Expressing Personalities of Conversational Agents through Visual and Verbal Feedback

Abstract

As the uses of conversational agents increase, the affective and social abilities of agents become important with their functional abilities. Agents that lack affective abilities could frustrate users during interaction. This study applied personality to implement the natural feedback of conversational agents referring to the concept of affective computing. Two types of feedback were used to express conversational agents’ personality: (1) visual feedback and (2) verbal cues. For visual feedback, participants (N = 45) watched visual feedback with different colors and motions. For verbal cues, participants (N = 60) heard different conditions of agents’ voices with different scripts. The results indicated that the motions of visual feedback were more significant than colors. Fast motions could express distinct and positive personalities. Different verbal cues were perceived as different personalities. The perceptions of personalities differed according to the vocal gender. This study provided design implications for personality expressions applicable to diverse interfaces.

1. Introduction

Future artificial intelligence (AI) assistants must be more than just question-and-answer machines. Computers need to express and recognize affect and emotions because affective computers can help reduce user frustration during interactions and enable smooth communication between users and computers [1]. Computers with unnatural expressions and reactions could deter human–computer interaction (HCI) and frustrate users. For instance, when users expect their conversational agents to deliver factual information without intense emotions, but conversational agents speak with intense emotions, users will become frustrated. Therefore, the feedback that matches with users’ perceptions of natural human-computer interaction needs to be designed.

Most studies in the field of affective computing have applied emotion to convey an agent’s internal states naturally. However, this research applied the concept of personality to implement more natural feedback for a conversational agent for the following reasons: (1) The concept of personality can allow individualized and complex features to be conveyed simultaneously; (2) it is possible to predict users’ perceptions by giving a personality to conversational agents; and (3) with personality, agents can implement consistent patterns of reactions rather than simple and immediate reactions.

Personalities are not innate for computer systems or interfaces; thus, humans must program them. In addition, they cannot be defined in a single sentence; instead, they are combinations of abilities, beliefs, preferences, dispositions, behaviors, and temperamental features with diverse behavioral and emotional attributes [2,3]. Therefore, the study posits the following general research question: RQ. How can conversational agents effectively express personalities?

Personalities can be expressed with two factors: (1) behavioral features (gestures, movements) and (2) verbal traits (voice, speech style). People with different personalities demonstrate different amplitudes and speeds of gestures and movements. Extroverts demonstrate faster, wider, and broader movements than introverts [4]. In addition, extroverts tend to demonstrate more reactive and faster movements and body gestures than introverts [5]. Personality and verbal traits are also highly interrelated. Extroverts tend to express more emotionality with positive emotions and use fewer formal expressions with agreement and gratitude [6]. Extroverts also demonstrate shorter silences and use more positive words and informal expressions than introverts. Introverts use more abstract words and formal words than extroverts [2,7].

Gestures are difficult to implement with current conversational agents because most of them are designed in the form of AI speakers. Existing conversational agents in the current market, including Amazon Alexa and Google Home, are not able to implement gesture movements. Instead of using gestures, AI speakers deliver simple visual feedback through a smart display in response to voice commands. Considering the current form of conversational agents, visual feedback was chosen as the personality expression element. As people with different personalities demonstrate different gesture speeds, the lighting speed could also be perceived as different personalities. Quick lighting is perceived as more active than slow lighting; therefore, the study posits the following research question: RQ 1. Can different visual feedback be perceived as different personalities?

Most current AI speakers use a consistent voice with the same speech style. As extroverts and introverts demonstrate different verbal traits, different verbal traits are perceived as different personalities. Therefore, the study posits the second research question: RQ 2. Can different verbal cues be perceived as different personalities?

Unlike previous studies, the current study applied a wider range of personalities rather than focusing on expressing only two contrasting personalities, such as extroversion and introversion. In addition, personality was expressed with diverse factors, including visual feedback with different colors and motions and five verbal cues, rather than focusing on a single element.

2. Related Work

2.1. Expressing Internal Machine States in HCI

People treat computers as social actors [8]. Considering the uncertainty of systems, it is difficult to set absolute standards for expressing a machine’s psychological and emotional states. In the field of HCI, machines’ internal states can be represented by diverse elements.

Verbal cues and text are the most direct ways of representing machines’ and computers’ internal states. Voice-related factors, such as volume, frequency range, and speech rate, are associated with the internal states of machines and computers [9,10]. Non-verbal behaviors, such as facial expressions, interpersonal closeness, gestures [9], and gaze [11,12], of embodied conversational agents were adjusted to demonstrate their social competencies in conversations. In particular, the gaze of the conversational agents were used to deliver back-channel feedback in turn-taking situations [13]. Diverse colors and lights were also used to express the social robots’ emotions [14,15,16], and a simple blinking light was used as a subtle expression to smooth human–robot speech interactions [17].

A large number of studies have focused on emotion [18]. This study argues that, for long-term HCI, the interface’s personality plays a more crucial role than simple emotional reactions. The current approach for personality expression was adapted from previous affective computing [19] and expressive output modality studies [15,16]. For instance, similar to prior studies, this study adopted cues to express conversational agents’ internal states, such as personality. Affective computers can both perform better when assisting humans and enhance their decision-making skills.

2.2. Personality Expressions of Computers and Interfaces

Personality can be defined as consistent patterns of feeling, thinking, and behavior [3]. In this study, personality was chosen to express the internal states of conversational agents for the following reasons: First, personality can easily express subtle and complex dimensions in individual differences. Second, personality can provide a wide range of predictions concerning how people may respond to various personalities [8]. In other words, people unconsciously use personality as a tool for assessing social partners [20]. In addition, much literature supports that individuals assess computers according to personality [8,21,22]. Lastly, personality is essential [23,24] in that it can help agents build natural relationships with humans.

Personality can be expressed and perceived in diverse ways because diverse viewpoints exist regarding personality. Personality could be seen as only measurable consistencies in behavior, or it could be seen as perceived consistencies regardless of its actual measurable consistencies. Among many diverse perspectives, the current study argues that personality is expressed and perceived through the interactions between the expectations of observers and the behaviors of the observed [20].

Most prior studies on a machine’s expressions of personality in particular have focused specifically on expressing two contrasting personalities, such as extroversion/introversion or dominance/submissiveness. Comparatively, few studies of personality expression have applied diverse types of personalities, such as openness and conscientiousness. Among the diverse categorizations of personalities, the Big Five personality traits were applied because they have demonstrated high reliability and validity in psychology and communication studies [25,26,27].

2.3. Combination of Diverse Cues

The two representative environmental factors in affective computing are (1) visual and behavioral elements (including color, light, video, and animations) and (2) audio stimuli (including noise, sound, and music). The current study uses both visual and verbal feedback.

Previous studies effectively examined personalities using diverse interfaces and contexts. Behavioral and linguistic factors and their correlations with personality were observed in virtual environments [28]. A social robot using different postures, gestures, and eye gazes depending on its personality and that of its user was also studied [9,29]. Linguistic cues and their correlations with personality [8] were also studied. LED lights [30] were used for the internal expressions of interfaces.

Personality and verbal traits, such as voice pitch, speech speed, wordiness, questioning, and vocal emotionality are highly related. Pitch and speech speed are two key factors of vocal communication [31] for delivering individual features. Vocal emotionality, which was expressed as the emotional reactions and feedback of the conversational agents in this study, was selected because the agent generating the desired emotional state and personality is important in users’ perceptions of the agent [1]. The questioning element was chosen because the usages of questionnaires determine the perception of voices [31]. Wordiness was chosen with reference to the natural language generator named PERSONAGE [9], which uses verbosity as the element to express personality dimensions through language. Vocal gender could be an element of a verbal cue because it is meaningful for personal attributes [6].

Based on previous studies, the current study argues that the personality of interfaces must be expressed using combinations of diverse cues. Therefore, both visual feedback and paralinguistic and verbal cues were adopted to express the conversational agent’s personality.

2.4. Big Five Personality

A number of researchers have examined and established human personality measurements. The Big Five Personality traits are a taxonomy for personality traits [32]. The Big Five personality model classified human personalities into five dimensions. Researchers agree that there are five robust factors of personality that can serve as a meaningful taxonomy for classifying personality attributes [33]. The first dimension is extraversion/introversion, which is associated with being sociable, talkative, and active. The second dimension is related to emotional stability and neuroticism. It is associated with being anxious, depressed, angry, and insecure. Agreeableness and likeability are the third dimension, which is associated with being trusting, good-natured, cooperative, and forgiving. The fourth dimension is conscientiousness, which is related to being careful, thorough, hardworking, and organized. The last dimension is openness to experience, which is related to being imaginative, cultured, intelligent, and artistically sensitive.

Diverse previous studies have applied the Big Five personality taxonomies. Researchers mapped human verbal traits to corresponding nonverbal and verbal behaviors of robots based on the extraversion–introversion personality dimension, which is one element of the Big Five personalities [9]. Another study investigated the role of a robot’s personality in the hands-off therapy process focusing only on the extraversion and introversion personality dimension [34]. A previous study examined the role of task context in perceived social robot personality applying the Big Five personality traits [10,35,36]. They found that extraverted people preferred robots with a similar personality when the robot was a tour guide [36]. Most previous studies that applied the Big Five personality traits focused on the extraversion/introversion dimension. In contrast to previous studies, this study adopted more diverse dimensions of the Big Five personality model for the conversational agent interface.

3. Methods

3.1. Overall Experimental Procedure

A total of 105 participants were recruited for the entire study (

Table 1. Number of participants depending on conditions.

Conditions	Number of Participants		Total
Visual feedback	Male	20	45
	Female	25
Verbal cues	Male	34	60
	Female	26

). All participants were recruited from a local community group located in Seoul, South Korea. They were all college students with a similar educational background, and they all had an intermediate level of English. In addition, they had similar residences and similar economic backgrounds.

Participants entered the room and received oral instructions about the study. Divided into groups, they were exposed to different experimental materials (visual feedback or verbal cues) because this research aims to express conversational agents’ personalities both with visual feedback and verbal cues.

A total of 45 participants (20 male, 25 females, mean age = 25, standard deviation of age = 3.6) were recruited for visual feedback. Divided into eight groups, five or six participants watched one visual feedback among eight visual feedback (see Section 3.2). A total of 60 participants (34 male, 26 females, mean age = 23.5, standard deviation of age = 2.5) heard 24 different verbal cues. Divided into five groups, 12 participants heard one condition of verbal cues among five conditions. (see Section 3.3).

After being exposed to experimental materials, they answered survey questions asking which personalities best matched the shown visual feedback and verbal cue voices. The survey questions were the short version of the Big Five personality questionnaires [37] (

Table A1. Survey questionnaires of Big Five Personality used in the experiment.

Item Number	Questionnaires
	How well do the following statements describe the personality of the experimental material? (1 = Disagree strongly, 7 = Agree strongly)
1	Is reserved.
2	Is generally trusting.
3	Tends to be lazy.
4	Is relaxed, handles stress well.
5	Has few artistic interests.
6	Is outgoing, sociable.
7	Tends to find fault with others.
8	Does a thorough job.
9	Gets nervous easily.
10	Has an active imagination.

Note. Scoring of the Big Five Inventories 10 scales are as follows (R= items are reversed-scored): Extraversion 1R, 6; Agreeableness: 2, 7R; Conscientiousness: 3R, 8; Neuroticism: 4R, 9; Openness: 5R, 10.

). The Big Five personality traits were applied because they have demonstrated high reliability and validity in psychology and communication studies [24,25,26,27]. In total, 10 questions were asked of participants in the survey.

3.2. Experimental Materials: Conversational Agent

A telepresence robot was used during the entire study because it is suitable for demonstrating both visual feedback and verbal cues (Figure 1). The visual feedback was demonstrated to participants through the telepresence robot’s screen. The verbal cues were played through the speaker of the robot. The static height of the telepresence robot is between 119.36 cm and 149.86 cm. The weight is 6.35 kg including the iPad with 7 kg. The height of the telepresence robot was fixed as 125 cm.

3.3. Experimental Materials: Visual Feedback

The main object of visual feedback is abstract moving circles (Figure 2). The background of the visual feedback was white blank space. The design of the visual feedback was motivated by diverse traits of the Big Five personalities. Considering that the Big Five personality traits are measured with bipolar adjectives [32], colors with bipolar traits were chosen based on previous studies about colors. Visual feedback was designed with combinations of bipolar or consistent colors. When using multiple colors, colors were only partially blended and overlapped, not entirely mixed. The partial overlapping of colors was motivated by the concept of continuity since each personality is defined as continuous traits between bipolar adjectives.

Among eight visual feedbacks, four visual feedbacks were designed with a single color (Visual feedbacks 2 and 5–7) and the other four visual feedbacks (1, 3, 4, and 8) were designed with combinations of two or three colors (

Table 2. Details of experimental materials for visual feedback. Five visual feedback were designed with different motions and colors.

Visual Feedback.	Motions	Motion Speed	Colors	Number of Colors
1	Moderate	0.25	Red + Yellow	2
2	Slow	0.15	Purple	1
3	Fast	0.35	Dark Blue + Yellow + Red	3
4	Moderate	0.25	Green + Blue	2
5	Slow	0.15	Grey	1
6	Moderate	0.35	Red	1
7	Slow	0.15	Cyon	1
8	Fast	0.35	Dark Blue + Yellow + Purple	3

). Both red and yellow reflect exciting, stimulating, and enlivening properties [38,39]. Considering this, visual feedback 1 was designed with combinations of red and yellow expressing an active personality. Visual feedback 6 used only red to observe the single impact. In contrast to visual feedback 1, visual feedback 2 was designed with a single purple color, which is related to vigorous and unhappy properties [38,40]. Visual feedback 3 combined red, yellow, and dark blue, which are colors with contrasting traits. Dark blue is related to soothing and tender properties [38], but red and yellow reflect active properties [38,39]. Visual feedback 4 was designed with green and blue, the opposite of visual feedback 3. In contrast with active colors, both green and blue are related to calm, peaceful, and comfortable properties [41]. Visual feedback 5 was designed with a combination of black and white. Black is related to melancholy and unhappy properties [42], while white is relaxing [15].

Motion was chosen as the design element of visual feedback because the five personalities have distinctly different levels of expressivity, proactivity, and energy [43]. The radius of circles’ movements was about 4 cm, and circles move clockwise and counterclockwise. All visual feedback moved with the same sequence (Figure 3). The duration of an object’s movement was 30 seconds. The motion size was also adjusted in parallel with the motion speed. The motion speed was adjusted using the Adobe Premiere Pro and After Effects programs, and the movement speed was adjusted between fast (0.35), moderate (0.25), and low (0.15) with the program. Visual feedback with fast motion demonstrated wider movements. Motions were selected with the opposite color traits.

3.4. Experimental Materials: Verbal Cues

Each script was written based on small talk from the chatbot Mitsuku [2] and current AI speakers such as Google Home and Amazon Alexa. Content that is commonly provided by current AI speakers and chatbots as the default setting was applied to observe the impacts of verbal cues, avoiding a heavy influence from content. Two Oddcast TTS programs [3] were used for vocal manipulations. Different voices were designed using the PC internal recorder with different scripts. The manipulation of each verbal cue occurred as follows (

Table 3. Experimental materials of verbal cues.

Script	Verbal cues
1: Introduction	3 Pitch (high/moderate/low) x 2 Gender
2: Weather conditions	2 Wordiness (high/low) x 2 Gender
3: Give directions	3 Speed (high/moderate/low) x 2 Gender
4: Ask a name	2 Emotionality (high/low) x 2 Gender
5: Scheduling	2 Questioning styles (high/low) x 2 Gender

).

The pitch level was adjusted using the Audacity program and Oddcast TTS. Using the Amazon Polly female and male voices as the default voices, the vocal pitch level was adjusted. The effects of vocal pitch on the evaluation of a social robot were examined [44]. Exuberant and calm voices were manipulated with different pitch levels. In the case of the male voice, the high pitch was 125.5 Hz, the moderate pitch was 110 Hz, and the low pitch was 98 Hz. In the case of the female voice, the high pitch was 226 Hz, the moderate pitch was 213 Hz, and the low pitch was 200 Hz. The pitch level was adjusted based on previous studies about pitch and personality [31,44]. Script 1 asked a social robot to introduce itself.

Wordiness was manipulated with two levels: high and low. The wordiness was adjusted with the length of the conversational agent’s answers. High wordiness was manipulated as more than 20 words, and the low wordiness condition was manipulated as less than 20 words. Script 2 asked for the name of the robot.

The speed was manipulated with three levels using the Audacity program. For female voices, the high speed was 18.5 times and the low speed was −11 times. For male voices, the high speed was 20 times and the low speed was −11 times. Voices with a moderate speed were not adjusted by the program. Script 3 asked the robot about the weather conditions.

Vocal emotionality was manipulated with two levels. For voices with low emotionality, a default voice was used. For voices with high emotionality, emotion functions were applied through the Oddcast program. Emotional expressions such as "Aha" and "Wow" were included. The Oddcast program without emotion was applied for voices without emotionality. Script 4 gave directions. The participant asked the agent for directions from their home to the subway station.

The questionnaire condition was manipulated with two levels: high and low. The high-level questionnaire was expressed in the format of interrogative sentences. The low-level questionnaire was expressed in the format of declarative sentences.

4. Results

4.1. Visual Feedback

One-way ANOVA and Tukey post-hoc tests were conducted. The following are the detailed statistical results of the one-way ANOVA (

Table 4. Results of visual feedback. Mean values of conversational agent’s perceived personalities depending on different visual feedback.

Personality Traits	Visual Feedback	Mean	SD	df	F Value	P Value
Agreeableness	1	5.83	0.75	4	14.37	0.00 ***
	2	2.66	1.21
	3	3.66	0.81
	4	6.16	0.75
	5	3.33	1.37
	6	4.20	1.25
	7	3.90	0.83
	8	3.70	1.08
Conscientiousness	1	3.50	1.05	4	6.12	0.00 **
	2	5.83	0.75
	3	2.33	1.37
	4	3.66	1.03
	5	3.33	1.86
	6	3.40	1.19
	7	4.20	0.83
	8	3.40	1.08
Extraversion	1	3.5	0.55	4	11.01	0.00 ***
	2	2.83	0.41
	3	5.90	0.89
	4	4.33	1.21
	5	3.33	1.21
	6	2.10	0.54
	7	3.60	1.94
	8	6.60	0.54
Neuroticism	1	2.00	0.89	4	23.15	0.00 ***
	2	6.16	0.75
	3	3.66	1.03
	4	2.83	0.75
	5	5.83	1.16
	6	3.2	0.83
	7	2.7	0.97
	8	2.5	0.79
Openness	1	3.16	1.21	4	3.24	0.03 *
	2	2.66	0.82
	3	3.16	0.75
	4	3.66	0.82
	5	2.16	0.75
	6	2.00	1.00
	7	1.80	0.83
	8	6.40	1.34

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). For agreeableness, visual feedback 4 (F (4,30) = 14.37, p < 0.001) demonstrated the highest mean value (M = 6.16). For conscientiousness, visual feedback 2 (F (4,30) = 6.12, p < 0.01) demonstrated the highest mean value (M = 5.83). For extraversion, visual feedback 8 demonstrated the highest mean value (Mean = 6.6) (F (4, 30) = 11.01, p < 0.001). For neuroticism, visual feedback 2 demonstrated the highest value (M = 6.167) (F (4, 30) = 6.167, p < 0.0001). For openness, visual feedback 4 demonstrated the highest mean value (M = 3.66) (F (4, 30) = 3.24, p < 0.05).

Based on the results of visual feedback (Table 4), we analyzed personality perceptions depending on the motions and colors of visual feedback. Our initial research purpose was to find personality perceptions depending on visual feedback, not the separate elements of visual feedback (colors, motions). Therefore, in the experiment, we did not separately test these two elements as individual factors. However, in the process of analysis, we discovered that levels of motions influenced the personality perceptions.

Depending on the levels of motions, eight visual feedbacks could be categorized into three groups. Visual feedbacks 3 and 8 were designed with fast motions. Visual feedbacks 1, 4, and 6 were designed with moderate motions. Visual feedbacks 2 and 5 were designed with slow motions (Table 2). We calculated the average mean values of conversational agents’ perceived personalities for each personality depending on the levels of motions (

Table 5. Average mean values of conversational agent’s perceived personalities depending on different levels of motions of visual feedback.

Personality Traits	Motion speed (Visual Feedback)
	Fast (3, 8)	Moderate (1, 4, 6)	Slow (2, 5)
Agreeableness	5.51	5.39	2.99
Conscientiousness	2.86	3.52	4.45
Extraversion	6.25	3.31	3.25
Neuroticism	3.08	2.67	4.89
Openness	4.78	2.94	2.20

).

Visual feedback with faster motions was more likely to be perceived as active and likeable personalities, such as extraversion and agreeableness. In the case of extraversion, fast motion showed the highest value (M = 6.25) compared with moderate (M = 3.31) and slow motions (M = 3.25). Agreeableness demonstrated a similar tendency in that faster motions showed higher values. However, slow motions were more likely to be perceived as negative and anxious personalities, including conscientiousness and neuroticism (Figure 4).

4.2. Verbal Cues

To analyze the results of study 2, two-way ANOVA and Tukey post-hoc test were conducted. Female voice with slow speed (M = 5.87, p < 0.05) and voice with low emotionality (M = 5.20, p < 0.005) were likely to be perceived as conscientious compared with other conditions (

Table A2. Results of two-way ANOVA for verbal cue conditions: Conscientiousness.

Verbal cues		Mean (SD)	df	F-Value	P-Value
Voice Gender	Female	4.62 (1.30)	1	0.12	0.72
	Male	4.75 (1.31)
Emotionality	High	4.17 (1.22)	1	8.69	0.00 ***
	Low	5.20 (1.16)
Gender * Emotionality	Female * high	4.29 (1.25)	1	1.16	0.29
	Male * high	4.04 (1.23)
	Female * low	5.45 (0.98)
	Male * low	4.95 (1.32)
Question	High	5.13 (1.15)	1	1.05	0.31
	Low	4.81 (0.97)
Gender * Question	Female * High	4.87 (1.22)	1	0.37	0.54
	Male * High	5.38 (1.07)
	Female * Low	4.75 (0.94)
	Male * Low	4.87 (1.05)
Speed	High	5.00 (1.56)	2	1.55	0.22
	Moderate	5.29 (1.08)
	Low	4.63 (1.76)
Gender * Speed	Female * High	5.21 (1.50)	2	4.81	0.01 *
	Male * High	4.79 (1.66)
	Female * Moderate	5.54 (0.62)
	Male * Moderate	5.04 (1.39)
	Female * Low	5.88 (0.86)
	Male * Low	3.37 (1.54)
Pitch	High	5.16 (1.20)	2	0.04	0.96
	Moderate	5.25 (1.25)
	Low	5.25 (1.19)
Gender* pitch	Female * High	5.71 (0.96)	2	0.79	0.46
	Male * High	4.63 (1.20)
	Female * Moderate	5.38 (1.15)
	Male * Moderate	5.13 (1.28)
	Female * Low	5.67 (0.86)
	Male * Low	4.83 (1.48)
Wordiness	High	5.33 (1.48)	1	3.2	0.09
	Low	5.16 (0.65)
Gender* Wordiness	Female * High	5.11 (0.4)	1	3.1	0.78
	Male * High	4.32 (0.23)
	Female * Low	4.53 (0.15)
	Male * Low	5.23 (0.42)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). Male voice that questions a lot (M = 4.62, p < 0.05), voice with high emotionality (M = 6.29, p < 0.001), and voice that speaking fast (M = 4.13, p < 0.01) were most perceived as extraversion (

Table A3. Results of two-way ANOVA for verbal cue conditions: Extraversion.

Verbal cues		Mean (SD)	df	F-Value	P-Value
Voice Gender	Female	4.72 (1.67)	1	0.13	0.72
	Male	5.14 (1.47)
Emotionality	High	6.29 (0.69)	1	8.96	0.00 ***
	Low	3.60 (0.92)
Gender * Emotionality	Female * high	6.33 (0.61)	1	1.68	0.2
	Male * high	6.20 (0.78)
	Female * low	3.95 (1.03)
	Male * low	3.25 (0.65)
Question	High	3.85 (1.57)	1	4.15	0.06
	Low	3.10 (0.98)
Gender * Question	Female * High	3.08 (1.54)	1	4.62	0.03 *
	Male * High	4.62 (1.22)
	Female * Low	3.12 (1.26)
	Male * Low	3.08 (0.66)
Speed	High	4.13 (0.81)	2	5.17	0.00**
	Moderate	3.16 (1.46)
	Low	3.17 (1.46)
Gender * Speed	Female * High	4.29 (0.81)	2	0.15	0.86
	Male * High	3.95 (0.81)
	Female * Moderate	3.42 (1.62)
	Male * Moderate	2.92 (1.31)
	Female * Low	3.54 (1.21)
	Male * Low	2.83 (1.19)
Pitch	High	4.13 (1.27)	2	2.45	0.09
	Moderate	3.27 (1.42)
	Low	3.60 (1.25)
Gender* pitch	Female * High	4.08 (0.76)	2	0.1	0.89
	Male * High	4.17 (1.67)
	Female * Moderate	3.54 (1.39)
	Male * Moderate	3.67 (1.17)
	Female * Low	3.38 (1.35)
	Male * Low	3.17 (1.54)
Wordiness	High	2.95 (1.30)	1	0.17	0.68
	Low	3.16 (1.15)
Gender * Wordiness	Female * High	2.83 (1.23)	1	1.49	0.76
	Male * High	2.45 (0.89)
	Female * Low	3.08 (0.54)
	Male * Low	3.21 (0.12)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). Female voice with high emotionality (M = 4.95, p < 0.05) were most perceived as openness (

Table A4. Results of two-way ANOVA for verbal cue condition: Openness.

Verbal cues		Mean (SD)	df	F-Value	P-Value
Voice Gender	Female	4.22 (1.76)	1	0.01	0.93
	Male	4.18 (1.65)
Emotionality	High	4.83 (1.35)	1	7.17	0.66
	Low	3.58 (1.78)
Gender * Emotionality	Female * high	4.95 (1.43)	1	0.19	0.01*
	Male * high	4.70 (1.32)
	Female * low	3.50 (1.81)
	Male * low	3.66 (1.83)
Question	High	4.42 (1.41)	1	0.35	0.56
	Low	4.19 (1.28)
Gender * Question	Female * High	4.79 (1.27)	1	0.03	0.87
	Male * High	4.04 (1.49)
	Female * Low	4.63 (1.11)
	Male * Low	4.79 (1.27)
Speed	High	4.08 (1.44)	2	0.03	0.97
	Moderate	4.04 (1.25)
	Low	3.98 (1.46)
Gender * Speed	Female * High	4.13 (1.28)	2	0.18	0.83
	Male * High	4.04 (1.64)
	Female * Moderate	4.29 (1.18)
	Male * Moderate	3.79 (1.32)
	Female * Low	4.25 (1.35)
	Male * Low	3.71 (1.57)
Pitch	High	4.44 (1.48)	2	0.66	0.53
	Moderate	4.29 (1.22)
	Low	4.00 (1.29)
Gender* pitch	Female * High	4.04 (1.46)	2	0.77	0.47
	Male * High	4.83 (1.45)
	Female * Moderate	4.29 (1.18)
	Male * Moderate	4.29 (1.32)
	Female * Low	4.04 (1.09)
	Male * Low	3.95 (1.51)
Wordiness	High	3.88 (1.83)	1	0.03	0.85
	Low	4.00 (1.52)
Gender* Wordiness	Female * High	3.87 (1.84)	1	0.25	0.62
	Male * High	4.00 (1.52)
	Female * Low	3.76 (1.23)
	Male * Low	4.12 (1.76)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). Male voice speaking fast (M = 3.50, p < 0.01) and low wordiness (M = 3.33, p < 0.001) were most perceived as neuroticism (

Table A5. Results of two-way ANOVA for verbal cue condition: Neuroticism.

Verbal cues		Mean (SD)	df	F-Value	P-Value
Voice Gender	Female	2.93 (1.14)	1	0.22	0.64
	Male	3.08 (0.96)
Emotionality	High	2.97 (1.02)	1	0.04	0.83
	Low	3.04 (1.09)
Gender * Emotionality	Female * high	2.87 (0.97)	1	1.33	0.26
	Male * high	3.08 (1.10)
	Female * low	3.29 (0.94)
	Male * low	2.79 (1.21)
Question	High	3.19 (1.39)	1	0.01	0.91
	Low	3.15 (0.99)
Gender * Question	Female * High	3.58 (1.56)	1	0.48	0.49
	Male * High	2.79 (1.12)
	Female * Low	3.29 (1.09)
	Male * Low	3.00 (0.90)
Speed	High	3.52 (1.49)	2	2.79	0.07
	Moderate	2.87 (1.29)
	Low	2.73 (0.97)
Gender * Speed	Female * High	3.50 (1.31)	2	0.08	0.00 **
	Male * High	3.54 (1.72)
	Female * Moderate	2.67 (0.98)
	Male * Moderate	2.79 (1.01)
	Female * Low	2.76 (1.03)
	Male * Low	3.00 (1.55)
Pitch	High	3.02 (1.13)	2	0.91	0.46
	Moderate	2.56 (1.25)
	Low	2.96 (1.38)
Gender* pitch	Female * High	2.96 (0.86)	2	0.07	0.92
	Male * High	3.08 (1.38)
	Female * Moderate	2.42 (1.26)
	Male * Moderate	2.71 (1.27)
	Female * Low	2.75 (1.25)
	Male * Low	3.17 (1.53)
Wordiness	High	2.67 (1.07)	1	3.2	0.00 ***
	Low	3.33 (0.72)
Gender * Wordiness	Female * High	5.33 (1.48)	1	2.2	0.78
	Male * High	5.24 (0.78)
	Female * Low	4.16 (0.65)
	Male * Low	4.77 (0.78)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). Agreeableness was excluded in the table because verbal cues condition did not show a significant value (

Table A6. Results of two-way ANOVA for verbal cue condition: Agreeableness.

Verbal cues		Mean (SD)	df	F-Value	P-Value
Voice Gender	Female	5.04 (1.39)	1	0.12	0.73
	Male	5.16 (1.08)
Emotionality	High	4.87 (1.20)	1	1.62	0.21
	Low	5.33 (1.25)
Gender * Emotionality	Female * high	4.96 (1.51)	1	0.66	0.42
	Male * high	4.79 (0.86)
	Female * low	5.12 (1.33)
	Male * low	5.54 (1.17)
Question	High	5.15 (1.28)	1	0.01	0.91
	Low	5.29 (0.87)
Gender * Question	Female * High	5.13 (1.13)	1	0.48	0.49
	Male * High	5.17 (1.46)
	Female * Low	5.38 (0.86)
	Male * Low	5.20 (0.92)
Speed	High	5.15 (1.66)	2	0.39	0.67
	Moderate	5.48 (1.19)
	Low	5.19 (1.39)
Gender * Speed	Female * High	5.50 (1.52)	2	0.19	0.83
	Male * High	4.79 (1.79)
	Female * Moderate	5.71 (0.99)
	Male * Moderate	5.25 (1.37)
	Female * Low	5.67 (1.13)
	Male * Low	4.71 (1.51)
Pitch	High	5.46 (1.21)	2	0.00	1.00
	Moderate	5.46 (1.09)
	Low	5.46 (1.25)
Gender* pitch	Female * High	5.42 (1.39)	2	1.49	0.23
	Male * High	5.50 (1.04)
	Female * Moderate	5.21 (0.96)
	Male * Moderate	5.71 (1.19)
	Female * Low	5.79 (0.84)
	Male * Low	5.13 (1.52)
Wordiness	High	5.50 (1.52)	1	2.19	0.15
	Low	4.71 (1.05)
Gender * Wordiness	Female * High	3.88 (1.84)	1	2.3	0.78
	Male * High	3.75 (0.98)
	Female * Low	4.00 (1.52)
	Male * Low	4.15 (0.23)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

).

5. Discussion

Considering the visual feedback results, different motions of visual feedback were highly influential on the perceptions of personalities. Fast, moderate, and slow motions could express different personalities, which answers the first research question of whether visual feedback with different motions could express different personalities. Regardless of colors, visual feedback with fast and moderate-speed motions were perceived as agreeable and kind (agreeableness). Visual feedback with slow motions was perceived as deliberate and careful (conscientiousness). Visual feedback with fast and moderate motions was considered to have a creative and imaginative personality (openness). Visual feedback with fast motions was considered active and sociable (extraversion). Visual feedback with slow motions was considered depressed and anxious (neuroticism). Different motions of visual feedback were perceived as different personalities.

The study suggested that fast motions were appropriate for expressing positive personalities such as agreeableness and extraversion. The traits of extroversion were related to ambition and sociability. Agreeableness can be interpreted as likability, cooperation, social conformity, and love [32], and these traits are more highly related to positive personalities than the other Big Five personalities. The results supported the previous finding that users’ perceptions of extraversion increased as the motion level increased [45]. Considering that previous studies about motions were conducted with virtual characters’ behavioral movements, this study provides design implications for more diverse formats of appearance constraint agents. The study argues that using slow motions is suitable for personalities that are usually perceived as negative concerning the results of personality perceptions associated with slow motions. The results reveal that slow motions were perceived as neurotic and conscientious. According to the Big Five personality trait study [43], traits associated with neuroticism were highly related to emotional instability, anxiety, and insecurity. In addition, traits associated with conscientiousness were related to thoroughness and planning, which could be negatively perceived in relationships. Slow motions of visual feedback are required to design conversational agents with negative personalities, such as neuroticism and conscientiousness. This is similar to human–human conversations because actively reactive conversational partners are perceived as having more active and positive personalities than passive and sullen conversational partners [5].

The study demonstrated that color was not a valid factor for expressing different personality traits. Even though the experimental materials were designed with the viewpoint that colors have different traits, colors do not show any consistent or significant patterns depending on different personality traits. Even though the two studies used experimental materials with diverse colors, such as red, purple, green, and green with blue, consistent patterns of personality perceptions were only demonstrated depending on the motion speed, regardless of color.

The visual feedback results highlight the issue of color subjectivity and objectivity. The subjectivity and objectivity of colors are a highly controversial issue among color scientists. Based on the argument that color perceptions can be organized with objective standards and systems, the relationship between colors and personalities has been widely observed and studied, particularly in human psychology. For instance, personality testing systems, such as the Color Pyramid Test [46], exist to evaluate personalities depending on different colors. In addition, according to the theory of color [39], red, yellow, and orange are related to exciting and enlivening features, blue and purple are related to anxious and yearning features, yellow is related to anger, and black is related to depression. Bold colors are more suitable for expressing dominant personalities than submissive personalities [20].

The current study argues that colors are not suitable for expressing diverse personalities because personal preference is decisive in the perception of colors. The finding of this study that colors are not influential factors is supported by diverse color perception studies. Color eliminativism, which is the notion that physical objects are not colored at all and colors are perceived psychologically rather than objectively, supports the current study’s results [47]. In addition, the color researcher Jean-Philippe Lenclos’s study of geographical color demonstrated that colors could be explained and defined based on geographical, cultural, and geographical conditions [48], which is highly related to the ecological perspective on color [40]. The correlations between specific colors and psychological factors as neither objective nor complicated [49]; therefore, more sophisticated color schemes that reflect other design elements should be designed for future studies.

The results could suggest the possibility that the number of colors could influence personality perceptions rather than their hue. In particular, both visual feedbacks 3 and 8 demonstrated significantly high mean values compared with other visual feedback instances in extraversion. Considering that only visual feedbacks 3 and 8 used three colors, while the others used only one or two colors, we suggest that using diverse colors rather than a single color could be suitable for expressing extraversion. However, since this study cannot ignore the impacts of hues, this study’s results do not fully support this argument.

Perceptions of the conversational agent’s personalities differed according to vocal gender. Fast speaking speed was perceived differently depending on the vocal gender. When the male voice spoke fast, it was perceived as emotionally unstable and nervous, but the female voice speaking fast was perceived as sociable and outgoing. Sociable and outgoing personalities are considered more positive than unstable and nervous personalities [50]. Even though the same verbal cue was used, the personality perception differed depending on vocal gender. In particular, people tended to perceive female voices as more positive than male voices. This result is highly related to the previous study findings that people prefer female extraverted voices for assistive social robots for elders [43,51].

The limitation of this study was that the design standard for choosing the colors of the visual feedback was based on previous studies, which could be subjective. Perceptions of colors could be influenced by personal preferences and cultural backgrounds. The findings of this study could be restricted to subjective and personal perceptions. For future studies, participants’ characteristics, such as personalities and gender, could be added as variables. Another limitation is that we did not separately measure the personality perceptions depending on motions and colors because visual feedback was designed with the combination of these two elements. Based on these findings, with more detailed conditions, the impacts of motions could be measured separately for a future study.

6. Conclusions

This study has explored how a conversational agent’s personalities can be expressed. The current study provided a novel approach to implement the natural feedback of conversational agents with the concept of personality rather than emotion. In addition, rather than focusing on two contrasting personalities, the current study applied wider personality categories. The study sought to implement more natural feedback by applying more diverse personality expression cues.

Visual feedback was used as an experimental material to answer the first research question: "Can different visual feedback be perceived as different personalities?" The results demonstrated that different motions of visual feedback were highly influential on the perception of personality regardless of color. Visual feedback with different motions could express different personalities. Visual feedback with slow motion was perceived as depressed and anxious, while fast motion was perceived as active and sociable. Moderate motion was perceived as having a creative, agreeable, and imaginative personality. Slow motion was perceived as conscientious and neurotic.

Verbal cues were used as experimental materials to answer the second research question: "Can different verbal cues be perceived as different personalities?" The results demonstrated that different verbal cues were perceived as different personalities. Regarding conscientiousness, the male voice with low emotionality and the female voice with a slow and moderate speed were statistically significant. The conversational agent with a voice with high emotionality, the female voice with fast speech, and the male voice asking many questions were perceived as extraverts. The female voice with high emotionality was likely perceived as having an open personality. The male voice speaking fast with low wordiness was perceived as neurotic.

The overall results of the conversational agents could be applied to diverse interfaces designed with smart displays, such as AI speakers, social robots, cars, and Internet of Things environments. In addition, this study applied a wider range of personalities rather than focusing on two contrasting personalities. Furthermore, more diverse elements were applied in personality expressions rather than focusing on simple factors.