New Intelligent Computer Intrusion Detection Method Using Hessian Local Linear Embedding and Multi-kernel Support Vector Machine

Computer networks frequently collapse under the destructive intrusions. It is crucial to detection hidden intrusions to protect the computer networks. However, a computer intrusion often distributes high dimensional characteristic signals, which increases the difficulty of intrusion detection. Literature review indicates that limited work has been done to address the nonlinear dimension reduction problem in computer intrusion detection. Hence, this study has proposed a new intrusion detection method based on the Hessian Local Linear Embedding (HLLE) and multi-kernel Support Vector Machine (SVM). The HLLE was firstly used to reduce the dimension of the original intrusion date in a nonlinear manner. Then the SVM with multiply kernels was employed to detect the intrusions. A real computer network experimental system has been established to evaluate the proposed method. Four typical intrusions have been tested. The test results show high effectiveness of the new detection method. In addition, the new method has been compared with the single-kernel SVM with Local Linear Embedding (LLE) or Principal Component Analysis (PCA). The comparison results demonstrate that the proposed HLLE plus multi-kernel SVM can provide the best computer intrusion detection rate of 97.1%.


INTRODUCTION
Along with the rapid development of internet and the associated application networks, network security has become a prominent and tough problem, in particular, intrusions and attacks on computer network systems becomes more complex and diverse.Huge economic losses have been caused by the computer and network intrusions and attacks every year.Therefore, it is essential to detect the intrusions and attacks in time to prevent damages of computers and networks.
The diversity and the evolution of the intrusion viruses make it very difficult in detecting and identifying the undergoing network intrusion.From the early worm virus to the recent shock, shock waves and the panda incense viruses, the attacking objects almost include all computer system accessed to the internet.The attacking viruses will cost the system resources, manipulate data and steal the confidential information, leading to massive economic losses.American business magazine "Information Weekly" has published a survey on the network intrusion and indicates that a network attack happens every second in the global scope.In such a situation, network security has become an urgent and practical problem and received worldwide attentions.How to develop and use the existing security technology to protect all kinds of resources from damage is the hot spot in the research field of network security.Effective intrusion detection technology is the key issue to solve this problem.
The Denial of Service (DoS), Remote to Local (R2L), User to Root (U2R) and Probe or Scan (PoS) are the most representative computer intrusion types.These four computer intrusion types cover almost all known computer intrusion types.The biggest thing in common of these four intrusion types is that the high dimensions of the attack feature data.Usually, there are more than 41 features need to check when one aims to detect these computer intrusions.Hence, it is very difficult in identifying an intrusion.To overcoming this problem, some advanced data processing methodologies have been proposed.Among the machine learning is a very useful technology in the field of computer and network security.However, the computer intrusion feature data is always contaminated by background noise.In addition, the high dimensionality of the intrusion data increases the difficulties in the intrusion detection (Abadeh et al., 2011).Hence, it is crucial to remove the noise and reduce the intrusion features into a low dimensional space.As a result, the intrusion recognition can be enhanced.
Up to date, the Artificial Intelligence (AI) technology has been extensively used in the computer intrusion detection.Representative methods include the Artificial Neural Network (ANN) (Nabil et al., 2012), Fuzzy logic (Abadeh et al., 2011) and Support Vector Machine (SVM) (Vapnik, 1995), etc. Elshoush and Osman (2011) gave a comprehensive review on the intrusion detection using AI methods.Nabil et al. (2012) adopted the artificial neural networks to detect the network intrusions.Abadeh et al. (2011) used the fuzzy inference to detect intrusion in computer networks.Yi et al. (2011) employed the SVM for network intrusion detection.Hu et al. (2003) proposed an anomaly detection algorithm, based on robust Support Vector Machines (SVM), which can effectively detect the intrusions even if noise existed.The success of these intelligent methods relies on proper and sufficient pre-processing of the original image data.However, the ANN or the Fuzzy detection model often requires a large amount of training samples.It is not realistic in practice because sufficient training samples are usually difficult to be ensured.As a result, if not enough training is provided the detection ability of the ANN or Fuzzy model should decay.In contrast, the SVM does not have this shortcoming and performs well even with a very small amount of samples.Hence, SVM is competent for parametric and/or nonparametric statistics and is found to be powerful in computer network intrusion detection (Mukkamala and Sung, 2003).Nevertheless, SVM is original to be suitable for binary classification tasks, which means SVM is usually used in one-class problem.The computer network intrusions are always a multi-class problem.How to use the SVM to deal with different types of intrusions is very important to ensure the security of computer networks.Since the multi-kernel SVM is proven to be effective in the computer intrusion detection (Li et al., 2012a), it is worthy testing its outcomes in the multi-class recognition problem in the computer intrusion detection.
On the other hand, many problems in network security begin with the preprocessing of raw high dimensional signals.This kind of preprocessing aims to attain more useful representations of the high dimensional signals in the form of low dimensional information.As a result, the subsequent operations such as classification, de-noising and visualization, etc., can be handed in an easy way (Hinton and Sejnowski, 1999).The high dimension of the intrusion feature data not only increases the computation cost but the detection rate.So, before the intrusion detection using the SVM, it is crucial to reduce the dimension of the intrusion feature data.One of the most useful methods in the dimension reduction is the Principal Component Analysis (PCA) (Li et al., 2011a).PCA can project the high dimensional features into a low dimensional space and thus eliminate useless ones.However, PCA is a kind of linear reduction approach and hence performs not well for the nonlinear cases (Li et al., 2012b).In contrast to PCA, the Hessian Local Linear Embedding (HLLE) (David and Carrie, 2003) has strong ability of dealing with nonlinear dimension reduction (Goldberg et al., 2008).HLLE is a kind of manifold learning algorithm, specifically, it is an extension of the Local Linear Embedding (LLE) algorithm (Roweis and Saul, 2000).The HLLE can reduce a high dimensional feature space into a much lower dimension space by keeping the topological structure of the original feature space.As a result, the output of HLLE preserves the nonlinear properties of the original space.Although HLLE has been found to be very powerful in the application of image processing and medical pattern recognition, etc., little work has been done to address the computer intrusion detection.Hence, the outcomes of the HLLE should be evaluated in the application of computer intrusion detection.
To enhance the performance of computer network intrusion detection, the present work has proposed a new method based on the integration of HLLE and multi-kernel SVM.The HLLE was firstly used to reduce the dimension of the original feature space of the intrusion data in a nonlinear manner.Then the multi-kernel SVM acted as a multi-class classifier to recognize the types of the intrusions.Compared with Li's work (Li et al., 2012a), the proposed method in this study has conducted the nonlinear dimension reduction using HLLE.Thus, the presented work has better performance in the computer network intrusion detection.Experimental tests have showed high performance of the proposed method.

THE PROPOSED DETECTION APPROACH
Since there is an inner correlation between intrusion and its future data, which is often impossible to be described via analytical model, the SVM (Vapnik, 1995) is employed to learn their relationships.The concept of the kernel trick allows SVM to be able to find the decision function from small datasets.In this study the multi-kernel SVM is used for the intrusion detection.Moreover, to improve the generalization of the SVM model, the HLLE algorithm is introduced to reduce the dimension of the input feature data.The theories about the multi-kernel SVM and HLEE as well as the proposed intrusion detection processes are briefly described as follows: The Hessian Local Linear Embedding (HLLE): HLLE (David and Carrie, 2003) extends the continuum theory for LLE to recover efficient parameterization without requiring convexity.Given an intrusion feature space q<<p).It takes six steps for HLLE to achieve the dimension reduction (David and Carrie, 2003): It can be seen that HLLE has the same framework of LLE.The only difference is that HLLE adopts the Hessian transform to replace the local reconstruction weight matrix in LLE.For more details of HLLE, it can refer to David and Carrie's work (David and Carrie, 2003).
The Support Vector Machine (SVM): According to Vapnik's definition of SVM (Vapnik, 1995), the basic SVM can be described as: if sample set: is linear separable, then exist a hyperplane 0 x that makes arbitrary sample , ( ) x y satisfy the following condition: 1 ( 1) x b when y x b when y where, ω : The weight vector b : A constant Then the goal of SVM is to find the optimal hyper plane 0 x and make ( , ) b ω subject to the following convex quadratic optimization problem: when the sample set S is not linear separable, a slack variable ξ is needed such that Eq. ( 2) can be rewritten as: where, C is a constant.
To solve the convex quadratic optimization problem, the Lagrange multiplier method has been introduced to transform it into a dual problem.Then, the original convex quadratic optimization can be solved by the maximum of the following dual problem: where,α is the Lagrange multiplier.Thus, the optimal discriminant function can be expressed as: when the sample set S is nonlinear separable, the kernel function ( , ) (Vapnik, 1995) is used to replace the dot product in Eq. ( 5).Common kernel functions include polynomial kernel, Radial Basis Function (RBF) kernel and sigmoid kernel.The polynomial kernel is defined as: The RBF kernel is defined as: where, σ is a constant.
The sigmoid kernel is defined as: where, β is a constant.
In practical classification problem, the choice of the kernel function is very important.Proper kernel relates directly to the classified accuracy.However, the above kernels are efficient for their own scopes of application.To overcome this problem, the multiply

The computer intrusion detection:
The proposed processes of the computer intrusion detection are given as follows: • Preprocess the computer network intrusion feature data into required format.• Reduce the dimension of the intrusion feature data using HLLE and obtain the new low-dimensional nonlinear intrusion features.• Train the multi-kernel SVM using the reduced intrusion features.• Test the performance of the SVM model for the computer intrusion detection.
A diagram of the proposed intrusion detection method is illustrated in Fig. 1.

EXPERIMENT TESTS AND RESULTS
In order to evaluate the performance of the proposed computer intrusion method, experiment tests have been implemented in this study.Figure 2 shows the experiment principle.A mini computer network has been established to conduct the experiments.The computer network is composed of a linux server, a windows server, the web link, two linux host and four windows hosts.Some manual attacks have been simulated and tested using this experiment system.
In this study, the Denial of Service (DoS), Remote to Local (R2L), User to Root (U2R) and Probe or Scan PoS) are introduced into the experiment system to validate the new detection method.Thirty two attribute features are monitored and recorded for every intrusion.These features include the bytes issued from source to destination, the bytes from destination to source, (duration, teardrop, Neptune etc.There are 2,000 samples for each intrusion type and hence the total samples are 8,000. After the data acquisition, the HLLE is used to reduce the 32 intrusion features into 2 manifold features.In the HLLE processing, the neighbors' k selects 8.The dimension reduction results are shown in Fig. 3. To highlight the efficiency of HLLE, HLLE is compared with LLE and PCA herein and Fig. 4 and 5 show the dimension reduction results of LLE and PCA.It can be seen in Fig. 3 that the intrusion feature space has been grouped into 4 clusters with evident demarcations although there are some overlaps between R2L and U2R.However, in the dimension reduction of LLE in Fig. 4, cluster R2L mixes up with U2R.Thus, LLE only separates three groups from the original feature space.It is also noticeable in Fig. 5 that the inter-class is confused in R2L, U2R and Pos when the PCA algorithm is applied.Hence, only two groups have been clustered from the original feature space by PCA.In addition, it needs emphasize that the inner-class distance and inter-class distance of Fig. 3 are much bigger than that in Fig. 4 and 5.This means that the performance of the dimension reduction of HLLE is superior to the LLE and PCA.-----------------------------------------------------------------------Testing (%)  -----------------------------------------------------------------    After the dimension reduction, a new feature space F 2000×2 is obtained, where 2000 denotes the samples and 2 denotes the dimension.Then, the SVM is applied to learn the feature space and recognize the intrusion patterns.In this study, half of the feature space F 2000×2 is used to train the SVM and the rest is for testing.
As mentioned above, the multi-kernel SVM is adopted in the intrusion detection.Three kernels have been used for the multi-kernel SVM.Moreover, comparison of multi-kernel SVM and single-kernel SVM has been implemented.Table 1 shows the intrusion detection results using single-kernel SVM while Table 2 shows the intrusion detection results using multi-kernel SVM.It can be seen in Table 1 that the performance of different kernels in the single-kernel SVM is almost the same.Owing to the dimension reduction processing, the intrusion detection rate has been enhanced significantly.This is because the original intrusion feature space contains lots of redundant information that disturb the SVM detection.However, these redundant features and noise information have been eliminated by the dimension reduction processing and hence the intrusion detection rate has increased.What is more important, the dimension reduction efficiency of HLLE is better than that of LLE while LLE outperforms to PCA.This is because HLLE and LLE can preserve the nonlinear information of the original data and HLLE is an improved version of LLE.As a result, the HLLE plus SVM provide the best intrusion detection results.Proof of this statement can also be found in Table 2.In Table 2, the multi-kernel SVM has been tested.It can be seen in the table that the polynomial plus RBF kernels can provide the best intrusion detection rate of 97.1%.It also can be noticed that by the use of multiply kernels, the intrusion detection rate has been enhanced against to single-kernel SVM.

CONCLUSION
Computer networks make the modern society prosperous.However, they often suffer the intrusions.To protect the computer networks, a new intrusion detection method base on the integration of Hessian Local Linear Embedding (HLLE) and multi-kernel Support Vector Machine (SVM) is proposed in this study.The innovation of this new detection method lies on that it adopts the nonlinear data dimension reduction and flexible kernels to improve the generalization ability of the SVM.A series of experiments have been conducted in a real computer network system.The test results demonstrate that the HLLE can capture distinct features from the original intrusion data and the multikernel SVM can detect the intrusions precisely.Future study will evaluate the new method in practice application.

Fig. 1 :
Fig. 1: The diagram of the computer intrusion detection

Table 1 :
The intrusion detection results using single-kernel SVM

Table 2 :
The intrusion detection results using multi-kernel SVM Intrusion detection rate (%)