edu.ucar.opensky::articles:21960 UCAR/NCAR - Library OpenSky Support PO Box 3000 Boulder CO 80307-3000 opensky@ucar.edu http://opensky.ucar.edu/ homepage pointOfContact 2025-12-25T15:51:10.323736 ISO 19115-2 Geographic Information - Metadata - Part 2: Extensions for Imagery and Gridded Data ISO 19115-2:2009(E) https://n2t.org/ark:/85065/d7f47s15 Using K-means clustering to detect anomalous file removes 2018-09-01T00:00:00Z publication https://n2t.org/ark:/85065/d7f47s15 Anderson, William D. author Genty, Marc D. author UCAR/NCAR - Library publisher One of the purposes of a data archive is to preserve irreplaceable data for future studies and generations. There are a number of ways that data can be lost from an archive, including accidental or malicious deletion of data. While there is a lot of software that can check for specific known threats or problems on a system, detecting non-specific anomalous behavior, such as unusual file removal patterns, is harder. One approach to detecting this kind of problem is machine learning. Machine learning algorithms can build a statistical model of what constitutes normal behavior and then flag data points that are outliers. To help protect the 87 petabytes of data in the National Center for Atmospheric Research's data archive, we explored our file removal patterns and implemented a k-means clustering solution to detect anomalous file removes. This approach can also be used to detect other anomalies, such as operational inconsistencies. UCAR/NCAR - Library OpenSky Support PO Box 3000 Boulder CO 80307-3000 opensky@ucar.edu http://opensky.ucar.edu/ homepage pointOfContact PDF Text theme Resource Type 2016-01-01T00:00:00Z publication Copyright 2018 CSREA Press. None eng geoscientificInformation