Oil Palm Fruit Classifications by using Near Infrared Images

Non-destructive assessment on oil palm Fresh Fruit Bunch (FFB) variability provides information for convenient oil palm field management practices. In this study a destructive assessment, using a microstrip sensor technique was firstly used to determine oil palm FFB maturity stages. A sample of FFB fruitlets was taken from 50 oil palm trees. The statistical data of insertion loss obtained from microstrip sensor was then been compared with moisture content in wet basis that was obtained from oven drying to correlate with the FFB maturity stages. The images of the above FFB samples were taken at the field by using multispectral camera. These images were processed and Near Infrared (NIR) hue value was extracted as the maturity feature. A relationship between images data of NIR and microstrip sensor data was developed to determine the maturity stages. The maturity stages data was plotted in the GIS map. Farmers will be able to go to specific location of FFB that has the optimum maturity and carried out the harvesting operation efficiently.


INTRODUCTION
Oil Palm (Elaeis guineensis Jac Var.tenera) is one of the most important plantation crops in Malaysia.Malaysia is the current leading producer and exporter of palm oil in the world (Tan et al., 2009).According to Corley (2009) by 2050 palm oil demand will probably reached 240 Mt.In order to full fill such demand an improved field management system need to be considered especially in harvesting operation.Harvesting Oil Palm Fresh Fruit Bunch (FFB) at right stage of ripeness is critical to ensure optimum quality and quantity of oil production and thus profitability of the industry (Rajanaidu et al., 1988).In the standard operating procedure, maturity of Oil Palm FFB is highly related with numbers of loose fruits detached from the oil palm FFB.This is use as a sign to harvest the FFB and has a drawback because uncollected loose fruit is one of the factors contributed to the losses in Malaysian Oil Palm industries.Sime Darby Plantation (2008) reported for every uncollected loose fruit per tree per year, the company will potentially lose almost RM 11 million in income.Thus, a new kind of monitoring system is required to determine maturity stages of oil palm to avoid losses due to loose fruit (Osborne et al., 1992).
It is important to provide ripe fruits to mills for oil extraction in order to produce high quality crude oils.Large scale crude oil mills in Malaysia are still using human expert to grade and sort the maturity stages of oil palm FFB.It has cause human errors in grading of oil palm FFB at the mills.The overripe and unripe FFB in the mills has caused the low quality of oil extraction.Due to these limitations, optimum crude oil production cannot be achieved; consequently, some operation costs become a loss and burden to manufacturers.
Various techniques have been developed to measure the ripeness of oil palm fruits or FFB maturity stages.The traditional method to determine the ripeness of FFB is by color.Oil palm fruits are usually black in color when they are unripe and they show reddish brown when they reach maturity stages (Jaffar et al., 2009).Usually, experts used to sense the maturity stage of FFB on the oil palm through experiences.Labors gathered their experience in the field and adapt themselves to familiarize with the maturity stages of FFB.
The objectives of this research were to develop GIS system as a control strategy for farmer to schedule the harvesting process and to assess the suitability of the Tetracam multispectral camera in determining the FFB maturity stages.

LITERATURE REVIEW
Nowadays, the research on ripeness sensing of oil palm FFB becomes more and more popular.Stepwise Discriminate Analysis for color grading of oil palm was developed in 2001.This technology was based on palm oil quantification by using a computer vision model in order to grade oil palm FFB.It simply links the oil contents and color for FFB ripeness determination.However, this invention is too expensive to be provided.Abdullah et al. (2001) developed tomographic radar imaging that focused on image acquisition technology to map the moisture content of oil palm FFB.Wan Ishak et al. (2000) studied the Red (R), Green (G) and Blue (B) color vision system, where R band had better discriminating feature to determine the ripeness of FFB.In recent years, hyperspectral imaging was introduced in the ripeness estimation of oil palm FFB.Junkwon et al. (2009) developed weight and ripeness estimation from hyperspectral imaging and he compared the result with RGB color images for the same FFB.They concluded that hyperspectral images provided a more detailed reflectance data than RGB images and the accuracy of result was claimed quite high.
Application of microstrip sensor was introduced to oil palm industry in Malaysia by Khalid and Abbas (1992).This microstrip sensor emitted microwave to detect the moisture content of oil palm FFB.Khalid and Abbas (1992), used microstrip sensor to differentiate FFB maturity stages from 14 to 24 weeks after anthesis.They found out the relationship of moisture content of oil palm with the ripeness of these fruits.The research of microstrip sensor in detecting the maturity stage of oil palm had shown a satisfactory result and they concluded that microstrip sensor has proved effective in examining the ripeness of oil palm fruits The multispectral camera was used as an imaging system as a non-destructive method.In this research the determination of FFB maturity stages will be done by analyzing the NIR band of the multispectral images and compare with the result with established method by using microstrip sensor.

MATERIALS AND METHODS
This research was carried out to develop software based supporting system to assist farmers to harvest oil palm FFB at the optimum maturity stage.Samples of various FFB maturity stages ranging from 5 to 24 weeks after anthesis were collected.The maturity stages of the FFB were determined by human expert and compared with the Oil Palm FFB maturity measured by using microstrip sensor.The microstrip sensor was calibrated to correlate the moisture content of fruitlet sample with respective FFB maturity stages.
This study was area was on an 8 ha field with the age of five years old oil palm tree at University Putra Malaysia Oil Palm field.Multispectral imaging and microstrip sensor was used to examine the FFB maturity stages (from 5 th to 24 th Week after anthesis).FFB fruitlet together with images samples were collected from the field.The insertion loss data provided by microstrip sensing technique and moisture content that gained from oven method were recorded in GIS.A computer program was developed to analyze the multispectral images and correlate with microstrip data to determine the maturity stages of FFB.The oil palm field covers 466 oil palm trees, similar number of 466 points will be dotted as a map in Arc Map as shown in Fig. 1.These points represent each individual tree in the field.When data of a tree was collected in the field, the data can be simply collected and stored in the point features.All the trees in the field were labelled by a laminated tag for convenient references.
The field scouting mainly consists of tagging or numbering the trees, collecting fruitlet samples for microstrip sensing and taking the pictures of the fruit bunch by using multispectral camera.During the scouting activities, one FFB from each tree was selected.The FFB picture was taken by using TETRACAM multispectral camera, which consists of three bands, namely Red band (R), Green band (G) and Near Infrared band (NIR).From each selected FFB, three fruitlets from basal, apical and equatorial as shown in Fig. 2 were plugged as samples and the age of the sample was recorded at this stage.The samples underwent a process to determine its insertion loss using microstrip sensor and correlate with its moisture content.Determination of moisture content was conducted base on standard procedure published by MPOB (Malaysian Palm Oil Board).

Calculating insertion loss of fruitlets with microstrip sensor:
The microstrip sensor is the prototype that was developed by Khalid and Abbas (1992).The fresh mesocarp was cut off from the nuts and was blended into meshes.The water dielectric emitted a certain amount of microwave energy to allow certain amount of electric current to pass through the meshes building a voltage difference for the whole electrical system.Figure 3 shows the sensor been used based on prototype developed by Khalid and Abbas (1992).
The input voltage V 1 and differential voltage V 2 of the samples were recorded.The insertion loss was calculated using formula in Eq. ( 1): Oven dry method: The insertion loss is the variable used to compare the value of wet basis moisture content in unit of percentage (%).Then the maturity stages were estimated based on the value of moisture content that was obtained by oven-drying method.MPOB standard drying method was used to get the moisture content value.The sample was dried in oven for 18 h at 75°C as recommended by MPOB and dry weight was recorded.The wet basis moisture content were determined.

Analysis of pictures from multispectral camera (image processing):
The acquired 50 pictures were then processed by using 2 software namely TETRACAM Pixel Wrech 2 (TPW 2) and MATLAB.
The Tetracam Multispectral camera captures the images in RAW format.By using Pixel wrench (TPW 2) the RAW images were processed and store in BMP format.
Figure 4 shows the sample of image that was analyzed by using TPW2.The BMP images with RGB color index was processed and transformed into BMP HSV color index by using MATLAB.In this research two sets of images were acquired: The first set of the multispectral images was a selected FFB from single oil palm tree.They were captured during day time in random interval of time with a static camera and eight images were taken as samples.The second set of multispectral images of FFB consisting 50 images from different oil palm tree and with different maturity stages, were taken as samples.
Mapping the recorded data: The map was digitized using WGS 1984 Mercator geodetic system.The digitized points automatically formed an attribute table to record the latitude and longitude of each palm trees.In this study, the moisture content data in wet basis of the fruitlets sample and the insertion loss were stored in the attribute table.

Microstrip sensor:
The data of the insertion loss of the FFB taken from 50 samples was plotted in Fig. 5.The curve shows that the insertion loss value decreased as the FFB developed towards the optimum maturity stage.
The insertion loss decreased very gradual in region one (week 5 to 10).In this region the fruitlet start to develop.The insertion loss had small differences from week 14 to 19, because the oil synthesis begun in this region .This exhibited similar results as reported by Khalid and Abbas (1992).In region for the insertion loss keeps on decreased until it reached optimum maturity stage.
Oven dry method: Figure 6 shows the relationship between insertion loss (dB) and wet basis moisture content (%) of FFB.It clearly shows that the insertion loss reading decreased as the moisture content inside the mesocarp decreased.

Analysis of pictures from multispectral camera (image processing):
The first set of the multispectral images was meant to investigate the effect of light intensity on the oil Palm FFB.Eight images were acquired and a graph of NIR Hue value versus number of pixels of the images is shown in Fig. 7.
From this graph, it shows that Hue color index is able to eliminate the effect of changing light intensity in oil palm field natural environment.The second set of the images consists of 50 multispectral images with various maturity stages.The NIR hue values of the images were extracted and their Coefficients of Variation (CV) of each maturity stages were determined.Table 1 tabulated the information of the maturity stages and it's respective coefficients of variations.It showed that each maturity stages had a different NIR hue value as a feature to indicate it stages.
From Table 1, the lowest CV ranged from NIR hue value of 226 to 242.It was found that maturity information contained in this range of maturity stages but differ in the number of pixels.By comparing the number of pixels and respective NIR hue value, maturity stage can be determined.A set of consecutive NIR hue value ranging from 235 to 241 were selected to plot a standard reference chart to classify the maturity stages of the FFB.
The plotted lines were divided into four groups of maturity stages as in Fig. 8 to 11.The groups are based  on the four regions of maturity stages as was discussed in Fig. 5.The curve has a different gradient in each region.FFB at each region have different internal and external features between other regions.In determination of maturity stages of FFB, a combination of three features was used, namely NIR Hue color index, pixel count and insertion loss.

Maturity stages determination:
A set of oil palm FFB NIR hue data from 10 different trees namely P11, P17, P19, P23, P32, P37, P40, P43, P45 and P47 was taken to test the maturity stages classification using the reference graph line as in Fig. 8 to 11.These data was discarded during calculating the coefficient of variation

Samples, p
Maturity stage, week --------------------------------------------------------------- classification using Euclidean distance function as shown in Eq. ( 2): The maturity stage of the test data is the maturity stage having the shortest distance from the test data point.In the graph based classification, the test data was plotted with the group of data in the region having the nearest insertion loss value.The gradient of the lines was compared and it shows that the maturity stage of the test data is the maturity stage of the lines having minimum gradient difference between the test data line.As an example, the P11 was in week 5 maturity stage when measured using microstrip sensor.NIR hue value ranging from 235 to 241 of P11 was plotted together with NIR hue value in the group of week 5 to 8. The minimum gradient difference was calculated and it was found that P11 was nearer to week 8. Hence from graph based classification P11 was classified in week 8 maturity stage.The result was compared with Euclidean distance approach by comparing the pixels count.P11 was classify as week 5 because it was found to have smaller difference with pixels in week 5.The result of the graph based classification of P11 is shown in Fig. 12.The classifications of the rest of the data were tabulated in the Table 2.
Classification: As showed in Table 2, Euclidean distance method classified sample P11 in week 5 maturity stage as measured by using microstrip sensor.Four samples namely P17, P23, P37 and P45 were classified with an error of 1 week early and 1 week later.The rest of the data namely P32, P19, P43, P47 and P40 have an error ranged from 2 to 5 weeks.In graph based method, two samples namely P17 and P32 were classified in week 10 and 23 respectively as measured by using microstrip sensor.Four samples namely P23, P37, P40 and P45 were classified within ±1 week error.Four more samples namely P11, P19, P43 and P43 have an error ranged from 3 to 5 weeks.Both methods classified P19, P23, P37, P43, P45 and P47 in the same maturity stages.Graph based approach classify six samples with ±1 week error and performed better than Euclidean distance classification.
From the classification result, NIR hue value alone is not sufficient to find the correlation with the microstrip sensor data.These approaches need to be support by some other non destructive methods that can determine other features of oil palm FFB physical properties such as mean bunch sizes and mean fruitlets sizes.This data can be acquired by further analysis of the multispectral images, but that is not within the scope of this study.

DISCUSSION
In oil palm industries, the price of the FFB fluctuates depending on various parameters of which the important parameter is Oil Extraction Rate (OER).Well manage harvesting operation will increase the OER.To have a better oil palm plantation management, a comprehensive tool is required to get the information of oil palm plantation.Software based management of harvesting operation will be a practical solution.Development of software to determine the maturity stage of the oil palm FFB required an extensive amount of input data.The best solution to get the input data was from an imaging device.In this research, multispectral imaging device was used to collect the images of the oil palm FFB and develop the relationship with the microstrip sensor data to determine the FFB maturity stages.From the analysis it was found that NIR hue color index had a minimum light intensity error.
Destructive measurements of FFB maturity stages by using microstrip sensor gave reliable information of maturity stages.But it depends on human expert to calibrate the sensor.During forming the meshes from the oil palm fruitlets, it was observed and proven that oil palm fruits had three main phases of development, as mentioned by Thomas et al. (1971).For the first 4 weeks (first phase), the embryo, endosperm could not be visualized.The second phase started from the 9 th weeks and the fruitlets started to develop it's embryo, mesocarp, epicarp and endosperm.Due to this variation and uncommon external features it is very hard to find the relationship of multispectral image with insertion loss by depending only on NIR hue color index.
In Fig. 5 the area under the curve was divided in four regions representing four main groups of oil palm maturity stages.This confirmed what was stated by Thomas et al. (1971).It has been observed that insertion loss increased as moisture content increased.When FFB reached its ripe stage (which were from week 23 to 24), the moisture content became lesser for the development of its seed.The flesh became oilier to become a palm product.The water that should be a 'bridge' for electric current to pass the capacitor of microstrip system at maturity stage became less; therefore, the differential voltage, V2 became less causing the insertion loss (dB) became less too.Similarly, the water loss in oven became less if we apply the meshes to be evaporated for 18 h giving the less value of moisture content.However, it was observed that some of the insertion loss (dB) showed some inconsistent result.The reason for the error insertion loss was simply due to the differential voltage, V2.Even though when fruitlets shared the same age with others, its property could never be monogenic with the others due to the way it grows.Their food supply mainly comes from the leaves that are attached nearest to its FFB.When its leaves were shaded by canopy, the FFB will sensibly get less food due to lower rate of photosynthesis to produce sugar.Thus, the ability for the fruitlets to get matured will become more impeded if comparisons applied, even though they were of the same weeks old and visually had a common bunch shape, fruitlet color and size.
During acquisition of the multispectral images there were uncontrolled parameters.They were the tilting angle of the camera and distance of the bunch from the camera.This imposed further complicated measurement of other external features to support NIR hue value in establishing the relationship with the maturity stages as measured by microstrip sensor.Each multispectral image was processed and the spectral bands were separated and only NIR band was analyzed.NIR hue value against number of pixel graph was plotted.Each estimated maturity stage was plotted in the same estimated age.The images plotted with large deviation within the same maturity stage were discarded in the process of measuring the common NIR hue value.Common pixel value among all maturity stage was determined in the NIR hue value range of 235 to 242.The common NIR hue value was further grouped into 4 groups according to maturity stage corresponded to the region under the curve in Fig. 5.
These graphs were used to determine the maturity stage of FFB in NIR band image.To test the classification, a set of NIR image containing 10 images that were discarded from the previous grouping process were used as test data.Euclidean distance function was added as a second classification approach to compare the performance.From the analysis, the NIR hue value of 235 to 241 was extracted and plotted in the selected group.The insertion loss value and corresponding estimated maturity stage of the data were known from this group.These classification processes further clarify the maturity stage of the discarded data.Euclidean distance approach classifies 50% of the data with ±1 week error while graph based classify 60% of the data with ±1 week error.The error can be further reduce by properly calibrate the microstrip sensor.Both approaches correctly classify six samples data in the same maturity stage namely P23, P37, P43, P45 and P47.
Information from the classification process was used as the input to the GIS software.The ages of the FFB already known as well as the FFB location.By using this information, farmers or plantation managers can schedule the harvesting operation.Location based harvesting can be executed.Work force can be precisely calculated since the number of bunches that is ready to harvest is known.Monthly and annually yield can be easily predicted.Early detection of oil palm tree stressed can also be detected or monitored.

CONCLUSION
Software based supporting systems for harvesting operation have a great potential to increase the quality of plantation management as well as potentially increasing the income of the oil palm planters and Malaysian oil palm industry.Software based management system catalysed the good agricultural practices.Software based supporting system required a high quality of input data.From our findings, multispectral (NIR) imaging has a good potential as a device to acquire high quality data.By depending only on NIR hue value the classification had 50 to 60% accuracy.By combining with other oil palm FFB features the accuracy can be increased.Other method of classification will be further investigated to improve the classification process.

Fig. 1 :
Fig. 1: Result of study area map after digitizing the 466 oil palm trees in study area

Fig. 5 :
Fig. 5: Graph of insertion loss against week after anthesis

Fig. 7 :
Fig. 7: Pixels value versus number of pixels of FFB in different light intensity

Table 1 :
Coefficient of variation and their respective FFB maturity stages

Table 2 :
Maturity stage classification results