L. Linville¹ , N. Downey¹ , C. Young¹

¹Sandia National Laboratories

Abstract:

Machine learning is a powerful way to accomplish a variety of seismic processing tasks, outperforming state-of-the-art for many traditional discrimination techniques in terms of broad applicability and performance. However, self-reported certainty from learned models can be unreliable, especially for data with characteristics outside of training distributions. While these cases can be rare, or go unnoticed during controlled experiment set-up and testing, real-world situations, production settings, and monitoring objectives require reliable estimates of certainty for decision making. In this work, the goal is to focus on understanding the current limitations of certainty from model output on a simple seismic event discrimination task using deep convolutional model architectures. With an understanding of the limitations on our current methods, can we improve upon the trustworthiness of a model’s reported certainty through data augmentation, model adaptations, or external trust metrics?