Are human editors better than AI?
This paper, produced by Salesforce AI researchers, explores how expert editing can enhance the quality of large language model (LLM)-generated text, addressing idiosyncrasies and improving alignment in the writing process. It offers insights into the impact of LLMs on language evolution and emphasizes the importance of well-designed editing tools.
Key Insights for Educators
- The increasing reliance on LLM-generated content may lead to homogenized writing, potentially reducing linguistic diversity and individual voice.
- Expert editing tools aligned with best writing practices can promote clarity, precision, and stylistic diversity in AI-generated text.
- The study involved 18 MFA-trained creative writers, highlighting the necessity of diverse perspectives in evaluating AI writing.
- A comprehensive taxonomy of edit categories was developed, grounded in established writing practices.
- The LAMP corpus was created, containing over 8,000 fine-grained edits made by professional writers on AI-generated text.
- No significant differences in perceived writing quality were found across texts generated by different LLMs (GPT-4, Claude 3.5, Llama 3.1).
- Automated detection and rewriting methods showed promise but did not fully match human expert performance.
- Writers consistently preferred text edited by humans over both LLM-edited and unedited LLM-generated text.
- The editing process for AI-generated text may lack the personal investment typical of editing one's own work, affecting the quality of edits.
- Future research could explore broader writing genres to better understand the applicability of findings across different contexts.