Seven Blunders of SIV Testing (and a Guide)

  • Blunder 1: Assessing text-dependent and text-independent technology using one database
  • Blunder 2: Inadvertently entangling voice and knowledge in text-dependent evaluations
  • Blunder 3: Comparing SIV accuracy obtained from channel-matched and channel-mismatched tasks
  • Blunder 4: Comparing SIV accuracy of two systems each of which was trained and/or tested with different average duration
  • Blunder 5: Publishing selected "lab" results
  • Blunder 6: Quoting False Accept Rate without quoting False Reject Rate, and vice versa
  • Blunder 7: Lacking statistical significance

By: Jiri Navratil, David Nahamoo

Published in: RC25311 in 2012

rc25311.pdf


Questions about this service can be mailed to reports@us.ibm.com .