DEEP OC Massive Online Data Streams
Deep learning for proactive network monitoring and security protection.
Published by
DEEP-Hybrid-DataCloud Consortium
Created:
- Updated:
Model Description
The use case challenges proactive network monitoring for security protection of computing infrastructures [1]. It builds an intelligent module as a machine learning application leveraging deep learning modeling to enhance functionality of intrusion detection system supervising network traffic flows. Preserving historical data, cyber security for such centers can be enhanced in hybrid way [3,4,5] using machine learning techniques, especially when large IT infrastructures and devices products a huge amount of dataflows continuously and dynamically.
The principle of this deep learning module (as a part of the use case) is proactive time-series forecasting. It builds prediction models capable to produce a system behaviour estimation near future. The discrepancy between the prediction and the reality gives an indication of anomaly (i.e. anomaly detection).
The challenge of the solution is it aims to scalable edge technologies [4] to support extensive data analysis and modelling as well as to improve the cyber-resilience in hybrid combination in real-time with the building intelligence module using neural networks and deep learning.
Deep learning architectures [2] available in this module are: MLP, CNN, autoencoder MLP, LSTM, GRU, bidirectional LSTM, sequence to sequence LSTM, stacked LSTM, attention LSTM, TCN, and stackedTCN.
References
[1]: Nguyen G., Dlugolinsky S., Tran V., Lopez Garcia A.: Deep learning for proactive network monitoring and security protection. IEEE Access, 2020, Volume 8, ISSN 2169-3536, DOI 10.1109/ACCESS.2020.2968718.
[2]: Nguyen G., Dlugolinsky S., Bobak M., Tran V., Lopez Garcia A., Heredia I., Malik P., Hluchy L.: Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey. Artificial Intelligence Review, Volume 52, Issue 1, pp. 77-124, ISSN 0269-2821, DOI 10.1007/s10462-018-09679-z. Springer Nature, 2019.
[3]: Nguyen G., Nguyen, M., Tran, D. and Hluchy L.: A heuristics approach to mine behavioural data logs in mobile malware detection system. Data & Knowledge Engineering, Volume 115, pp. 129-151, ISSN 0169-023X, DOI 10.1016/j.datak.2018.03.002. Elsevier, 2018.
[4]: Bhattacharyya, D.K. and Kalita, J.K., 2013. Network anomaly detection: A machine learning perspective. Chapman and Hall/CRC.
[5]: Dua, S. and Du, X., 2016. Data mining and machine learning in cybersecurity. Auerbach Publications.
Test this module
You can test and execute this module in various ways.
Excecute locally on your computer using Docker
You can run this module directly on your computer, assuming that you have Docker installed, by following these steps:
$ docker pull deephdc/deep-oc-mods
$ docker run -ti -p 5000:5000 deephdc/deep-oc-modsExecute on your computer using udocker
If you do not have Docker available or you do not want to install it, you can use udocker within a Python virtualenv:
$ virtualenv udocker
$ source udocker/bin/activate
(udocker) $ pip install udocker
(udocker) $ udocker pull deephdc/deep-oc-mods
(udocker) $ udocker create deephdc/deep-oc-mods
(udocker) $ udocker run -p 5000:5000 deephdc/deep-oc-mods
In either case, once the module is running, point your browser to
http://127.0.0.1:5000/ and you will see the API
documentation, where you can test the module functionality, as well as
perform other actions (such as training).
For more information, refer to the user documentation.
Train this module
You can train this model using the DEEP framework. In order to execute this module in our pilot e-Infrastructure you would need to be registered in the DEEP IAM.
Once you are registedered, you can go to our training dashboard to configure and train it.
For more information, refer to the user documentation.