DEEP OC Massive Online Data Streams

Deep learning for proactive network monitoring and security protection.


Published by DEEP-Hybrid-DataCloud Consortium
Created: - Updated:

Module description

Build Status

The use case challenges proactive network monitoring for security protection of computing infrastructures [1]. It builds an intelligent module as a machine learning application leveraging deep learning modeling to enhance functionality of intrusion detection system supervising network traffic flows. Preserving historical data, cyber security for such centers can be enhanced in hybrid way [3,4,5] using machine learning techniques, especially when large IT infrastructures and devices products a huge amount of dataflows continuously and dynamically.

The principle of this deep learning module (as a part of the use case) is proactive time-series forecasting. It builds prediction models capable to produce a system behaviour estimation near future. The discrepancy between the prediction and the reality gives an indication of anomaly (i.e. anomaly detection).

The challenge of the solution is it aims to scalable edge technologies [4] to support extensive data analysis and modelling as well as to improve the cyber-resilience in hybrid combination in real-time with the building intelligence module using neural networks and deep learning.

Deep learning architectures [2] available in this module are: MLP, CNN, autoencoder MLP, LSTM, GRU, bidirectional LSTM, sequence to sequence LSTM, stacked LSTM, attention LSTM, TCN, and stackedTCN.

References

[1]: Nguyen G., Dlugolinsky S., Tran V., Lopez Garcia A.: Deep learning for proactive network monitoring and security protection. IEEE Access, 2020, Volume 8, ISSN 2169-3536, DOI 10.1109/ACCESS.2020.2968718.

[2]: Nguyen G., Dlugolinsky S., Bobak M., Tran V., Lopez Garcia A., Heredia I., Malik P., Hluchy L.: Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey. Artificial Intelligence Review, Volume 52, Issue 1, pp. 77-124, ISSN 0269-2821, DOI 10.1007/s10462-018-09679-z. Springer Nature, 2019.

[3]: Nguyen G., Nguyen, M., Tran, D. and Hluchy L.: A heuristics approach to mine behavioural data logs in mobile malware detection system. Data & Knowledge Engineering, Volume 115, pp. 129-151, ISSN 0169-023X, DOI 10.1016/j.datak.2018.03.002. Elsevier, 2018.

[4]: Bhattacharyya, D.K. and Kalita, J.K., 2013. Network anomaly detection: A machine learning perspective. Chapman and Hall/CRC.

[5]: Dua, S. and Du, X., 2016. Data mining and machine learning in cybersecurity. Auerbach Publications.

Test this module

You can test and execute this module in various ways.

Excecute locally on your computer using Docker

You can run this module directly on your computer, assuming that you have Docker installed, by following these steps:

$ docker pull deephdc/deep-oc-mods
$ docker run -ti -p 5000:5000 deephdc/deep-oc-mods

Execute on your computer using udocker

If you do not have Docker available or you do not want to install it, you can use udocker within a Python virtualenv:

$ virtualenv udocker
$ source udocker/bin/activate
$ git clone https://github.com/indigo-dc/udocker
$ cd udocker
$ pip install .
$ udocker pull deephdc/deep-oc-mods
$ udocker create deephdc/deep-oc-mods
$ udocker run -p 5000:5000  deephdc/deep-oc-mods

In either case, once the module is running, point your browser to http://127.0.0.1:5000/ and you will see the API documentation, where you can test the module functionality, as well as perform other actions (such as training).

For more information, refer to the user documentation.

Train this module

You can train this model using the DEEP framework. In order to execute this module in our pilot e-Infrastructure you would need to be registered in the DEEP IAM.

Once you are registedered, you can go to our training dashboard to configure and train it.

For more information, refer to the user documentation.

Categories

services, docker, api-v2, trainable, inference, deep learning, keras, tensorflow

License

License: Apache 2.0

Configure and train

Get the code

Github Docker Hub