skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering

Transactions of the Association for Computational Linguistics, 2019-11, Vol.7, p.387-401 [Peer Reviewed Journal]

2019. This work is published under https://creativecommons.org/licenses/by/4.0/legalcode (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. ;ISSN: 2307-387X ;EISSN: 2307-387X ;DOI: 10.1162/tacl_a_00279

Full text available

Citations Cited by
  • Title:
    Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering
  • Author: Wallace, Eric ; Rodriguez, Pedro ; Feng, Shi ; Yamada, Ikuya ; Boyd-Graber, Jordan
  • Subjects: Computers ; Datasets ; Human-computer interaction ; Information retrieval ; Language ; Linguistics ; Natural language ; Natural language (computers) ; Natural language generation ; Question answer sequences ; Questions ; User interface ; Writing
  • Is Part Of: Transactions of the Association for Computational Linguistics, 2019-11, Vol.7, p.387-401
  • Description: Adversarial evaluation stress-tests a model’s understanding of natural language. Because past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human- in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human–computer matches: Although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.
  • Publisher: One Rogers Street, Cambridge, MA 02142-1209, USA: MIT Press
  • Language: English
  • Identifier: ISSN: 2307-387X
    EISSN: 2307-387X
    DOI: 10.1162/tacl_a_00279
  • Source: ROAD: Directory of Open Access Scholarly Resources
    DOAJ Directory of Open Access Journals
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait