Email is vital in academic communication, especially for non-native English speakers. The rise of AI writing tools (e.g., ChatGPT) is reshaping communication, but their impact on email quality remains understudied.
As an experienced university instructor, your evaluation will provide key insights on:
- Effectiveness differences between AI-assisted and human-written emails
- Core features of high-quality academic emails
TASK: Evaluate 4 anonymized student emails (≈15 minutes)
In the following questions (5–8) , you will review student emails addressed to faculty members. Please read each message carefully and base your judgments on its content. For each email:
Note: There are no right or wrong answers—your personal perceptions are essential to this research.
Ethical Assurance
1. Years of teaching experience at university level: < 3 years 3-6 years ＞ 6 years
2. Primary academic discipline you teach: Humanities & Social Sciences STEM (Science/Tech/Engineering/Math) Business & Economics Arts & Design Other
3. How many STUDENT emails do you receive on average per week? 0–10 emails 11–30 emails 31–50 emails More than 50 emails
4. Have you encountered STUDENT emails that you suspected were AI-generated? Never Rarely (1-2 per week) Occasionally (3-5 per week) Frequently (over 5 per week)
5. Situation:
Student needs to schedule a project meeting with Prof. Adams during office hours but conflicts with regular times, so requests an alternative time.
Office Hours Appointment Request
Best regards,
Mike
Email Evaluation: Indicate if the email was written by AI or a human:
This email is AI-assisted.
This email is written by the student without any AI assistance.
This email is easy to understand.
The tone is polite for student-faculty communication.
If received, I would respond positively.
6. Situation: Student in LNG 01 Applied Linguistics declines Prof. Adams' research invitation due to a busy schedule and prior commitments.
Lily
Email Evaluation: Indicate if the email was written by AI or a human:
This email is AI-assisted.
This email is written by the student without any AI assistance.
This email is easy to understand.
The tone is polite for student-faculty communication.
If received, I would respond positively.
7. Situation: Student in LNG 01 Applied Linguistics declines Prof. Adams' research invitation due to a busy schedule and prior commitments.
Thank You for the Research Opportunity
Best regards,
Lily
Email Evaluation: Indicate if the email was written by AI or a human:
This email is AI-assisted.
This email is written by the student without any AI assistance.
This email is easy to understand.
The tone is polite for student-faculty communication.
If received, I would respond positively.
8. Situation:
Student needs to schedule a project meeting with Prof. Adams during office hours but conflicts with regular times, so requests an alternative time.
I am your student Mike. I want to schedule an appointment with you during your
office time in this week to discuss an upcoming project. However, I found I had another activity need to attend in your usual office time. So I want to know if we can alter the
time that works for both of us.
Email Evaluation: Indicate if the email was written by AI or a human:
This email is AI-assisted.
This email is written by the student without any AI assistance.
This email is easy to understand.
The tone is polite for student-faculty communication.
If received, I would respond positively.
9. When deciding whether an email was AI-generated or human-written, what specific features did you rely on most? (Select all that apply) Tone Patterns （e.g., politeness, formality, consistency of tone） Structural Elements（e.g., greeting, opening, closing） Grammatical Features（e.g., tense consistency, subject-verb agreement, lexical variety） Unusual/Awkward Expressions (e.g., word-for-word translation, misused slang or idioms) Others（If you pick this, please briefly explain below!）
10. To what extent do you view EMI university students’ use of AI-assisted tools (e.g., ChatGPT) for email composition as positive or negative? Completely negative Slightly negative Neutral Slightly positive Completely positive
11. Thank you for your kind help! Finally, may we contact you for an around-30-minute interview to further explore your email evaluation experience? Yes, I'm willing to participate. (- Contact method: ________ (Email/WeChat/Phone) - Preferred time: ________ (e.g., Weekday evenings) No, thanks.