All Content

Are human editors better than AI?

 This paper, produced by Salesforce AI researchers, explores how expert editing can enhance the quality of large language model (LLM)-generated text, addressing idiosyncrasies and improving alignment in the writing process. It offers insights into the impact of LLMs on language evolution and emphasizes the importance of well-designed editing tools.


View PDF


Key Insights for Educators

  • The increasing reliance on LLM-generated content may lead to homogenized writing, potentially reducing linguistic diversity and individual voice.
  • Expert editing tools aligned with best writing practices can promote clarity, precision, and stylistic diversity in AI-generated text.
  • The study involved 18 MFA-trained creative writers, highlighting the necessity of diverse perspectives in evaluating AI writing.
  • A comprehensive taxonomy of edit categories was developed, grounded in established writing practices.
  • The LAMP corpus was created, containing over 8,000 fine-grained edits made by professional writers on AI-generated text.
  • No significant differences in perceived writing quality were found across texts generated by different LLMs (GPT-4, Claude 3.5, Llama 3.1).
  • Automated detection and rewriting methods showed promise but did not fully match human expert performance.
  • Writers consistently preferred text edited by humans over both LLM-edited and unedited LLM-generated text.
  • The editing process for AI-generated text may lack the personal investment typical of editing one's own work, affecting the quality of edits.
  • Future research could explore broader writing genres to better understand the applicability of findings across different contexts.