MDTAIM
My Master's thesis: turns multidimensional time-series anomaly scores into transaction databases for itemset mining.
TL;DR
MDTAIM is my Master’s thesis project at Politecnico di Torino for detecting multidimensional anomalies in high-dimensional time series by converting multidimensional Matrix Profile scores into transaction databases and mining frequent, maximal, or closed itemsets. I built the Python pipeline end to end, from data preprocessing and anomaly score conversion to SPMF-based itemset mining and postprocessing. On real datasets, the best conversion method reached 0.85 F1 on MSCRED, 1.00 F1 on a photovoltaic system dataset, and 0.86 F1 on MGAB.
Problem
Most time-series anomaly detection methods return either point-level anomaly scores or a ranked list of unusual subsequences. That is not enough when the useful question is combinatorial: which dimensions were anomalous together, how often did that pattern repeat, and which subsets form the strongest multidimensional event?
MDTAIM reframes that output as an itemset mining problem. Each time window becomes a transaction, each anomalous dimension becomes an item, and the query layer can ask for repeated, maximal, closed, or high-utility anomaly patterns instead of only a top-k anomaly list.
Approach
The pipeline starts by computing a Matrix Profile for each dimension, producing an anomaly-score matrix over time. I then convert that continuous score matrix into fixed-width transactions: a time window is one transaction, and a dimension is included when its score pattern crosses the selected conversion rule.
The downstream mining step uses SPMF algorithms to answer different query types. Frequent itemsets capture recurring multidimensional anomaly patterns, closed itemsets preserve exact support structure, and maximal itemsets suppress smaller subsets when a larger pattern explains them.
Key decisions
- Transaction mining over top-k ranking: chose itemset mining over only returning top-k multidimensional discords because the target queries needed repetition, closedness, and maximality constraints.
- Matrix Profile as the default scorer: chose Matrix Profile over forecasting or reconstruction models because it is unsupervised, subsequence-based, and needs one main domain parameter: the expected discord length.
- Windowed binary conversion: chose fixed-width transaction windows over point-level score aggregation because itemset algorithms need discrete transactions, while anomaly scores remain continuous and dataset-dependent.
- SPMF integration: chose SPMF over implementing mining algorithms from scratch because the thesis needed comparable frequent, closed, and maximal itemset algorithms under one command-line interface.
Hard parts
The hardest part was the conversion layer between continuous anomaly scores and discrete itemset inputs. A global score threshold was not reliable because Matrix Profile magnitudes changed across datasets, dimensions, and subsequence lengths, so I implemented and compared statistical, average-energy, and longest-nonzero-subsequence conversion rules.
The second hard part was avoiding duplicate detections caused by sliding windows. When the same discord appeared across adjacent windows, the pipeline kept the stronger window-level detection instead of letting one physical anomaly inflate the itemset support.
Results
- 0.85 F1 on MSCRED using the longest-nonzero-subsequence conversion rule, across 30 dimensions and 20,000 time points.
- 1.00 F1 on a photovoltaic system dataset with 22 dimensions and 1,511 time points.
- 0.86 F1 on MGAB, a 10-dimensional chaotic time-series benchmark with about 97,600 time points.
- Evaluated synthetic scaling experiments across 10 to 200 dimensions and 10,000 to 500,000 time points.
Stack
Python, Poetry, NumPy, pandas, Matrix Profile, SPMF, scikit-learn, Matplotlib, Plotly, PyYAML.