Credit Card Fraud Analysis
Analysis of credit card fraud data using neural networks.
The datasets contains credit card transactions over a two day collection period in September 2013 by European cardholders. There are a total of 284,807 transactions, of which 492 (0.172%) are fraudulent.
The dataset contains numerical variables that are the result of a principal components analysis (PCA) transformation. This transformation was applied by the original authors to maintain confidentiality of sensitive information. Additionally the dataset contains
Amount, which were not transformed by PCA. The
Time variable contains the seconds elapsed between each transaction and the first transaction in the dataset. The
Amount variable is the transaction amount, this feature can be used for example-dependant cost-senstive learning. The
Class variable is the response variable and indicates whether the transaction was fraudulant.
The dataset was collected and analysed during a research collaboration of Worldline and the Machine Learning Group of Université Libre de Bruxelles (ULB) on big data mining and fraud detection.
The data is split into 5 train/test sets, balanced to account for fraud being a relatively rare event. For each split, a multi-layer perceptron (MLP) neural network is fit consisting of 28 input nodes, a densely connected hidden layer with 22 nodes with S-shaped rectified linear activation and 20% dropout, and 1 output node with a sigmoid activation. The models are fit using batches of 1200 observations for up to 100 epochs, although validation loss (binary crossentropy) is monitored to permit early stopping. Stochastic optimization is performed using Adam.
The final model is an ensemble of the stratified k-fold neural networks, constructed by averaging the model predictions.
The models are implemented in Python using Keras and TensorFlow as the backend, although you could use Theano if you like (remember to remove the TensorBoard callback).
Run the model using the following command:
KERAS_BACKEND=tensorflow ipython src/model.py
The final model achieves an overall f1 score of 1.00, with 95% sensitivity (recall) and 19% precision for the positive class. That is, the model correctly identifies 95% of the fraud cases (true positives) but only 19% of the transactions predicted as fraudulent were actually fraudulent. The model catches 95% of the fraudulent cases — it could identify more cases of fraud but would then also have lower precision.
precision recall f1-score support 0.0 1.00 0.99 1.00 284315 1.0 0.19 0.95 0.31 492 avg / total 1.00 0.99 1.00 284807
Predictions 0 1 Truth 0.0 282274 2041 1.0 26 466
Precision Recall Curve
Andrea Dal Pozzolo, Olivier Caelen, Reid A. Johnson and Gianluca Bontempi. Calibrating Probability with Undersampling for Unbalanced Classification. In Symposium on Computational Intelligence and Data Mining (CIDM), IEEE, 2015 (PDF)
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization Published as a conference paper at ICLR 2015 (PDF)
This code repository is released under the MIT "Expat" License.