Statistical Parametric Speech Synthesis of Malay Language using Found Training Data

The preparation of training data for statistical parametric speech synthesis can be sophisticated. To ensure the good quality of synthetic speech, high quality low noise recording must be prepared. The preparation of recording script can be also tremendous from words collection, words selection and sentences design. It requires tremendous human effort and takes a lot of time. In this study, we used alternative free source of recording and text such as audio-book, clean speech and so on as the training data. Some of the free source can provide high quality recording with low noise which is suitable to become training data. Statistical parametric speech synthesis method applying Hidden Markov Model (HMM) has been used. To test the reliability of synthetic speech, perceptual test has been conducted. The result of naturalness test is fairly reasonable. The intelligibility test showed encouraging result. The Word Error Rate (WER) for normal synthetic sentences is below 15% while for Semantically Unpredictable Sentences (SUS) is averagely in 30%. In short, using free and ready source as training data can leverage the process of preparing training data while obtaining motivating synthetic result.


INTRODUCTION
Speech synthesis is a process of converting text representation of speech into waveform that can be heard by listeners (Ekpenyong et al., 2014).Statistical parametric speech synthesis (Zen et al., 2009) is a method of using natural speeches and texts as training data, the input training data is transformed into intermediate label data and the speech synthesizer uses the intermediate label data to synthesize speech.This method is using famous mathematical model which is Hidden Markov Model (HMM) (Ibe, 2013) that can be applied in various area such as pattern recognition, signal processing and so on.The quality of the synthetic speech is affected by the quality of the training data.Therefore, the preparation of input training data is crucial and requires thorough design of script and good quality of recording.However, the process of preparing input training data is not an easy task.The selection of script requires tremendous human effort in collecting words and designing sentences (Tan and Salleh, 2009).The recording setup must be good to reduce noise and able to record clean speech.
In this study, we have built a Malay language speech synthesizer using alternative sources such as audio-books, educational storytelling audio data, clean speech and so on.Those data can be obtained online for free.We have taken the free speech online and segmented only the clean portion and prepared the corresponding script to be the input training data.The synthetic speech using free source has been compared to the synthetic speech using specially designed and recorded training data.More details are explained in later section.

Statistical parametric speech synthesis using Hidden Markov Model (HMM):
Statistical parametric speech synthesis is a speech synthesis method which generates average sets of similar sounding speech segment instead of using real speech segment like in unit selection method (Lim et al., 2012).Typically, it uses mathematic model such as Hidden Markov Model (HMM) to model the spectral and excitation parameters extracted from a real speech database.Model parameters are usually estimated using Maximum Likelihood (ML) criterion as: (1) where, λ is set of model parameters, O is set of training data and W is set of word sequences corresponding to O. When we want to generate desired speech, first the sentences is composed, then follow the equation below: where, o is the speech parameters we want to generate w is the given word sequence and estimated models.These parameters are then used to generate speech waveform.Any generative model can be used but HMM is most widely used model in this approach because of its memory-less ability to reduce complexity during process.It is commonly known as HMM-based speech synthesis (Yoshimura

METHODOLOGY
Database preparation: The found data for from the website http://free-islamic-lectures.com which is a free resource providing Islamic teaching recording.It offers free download of audio recording of Al reading in Arabic language with translation of Malay language.We manually segmented the Malay speech portion out and prepared the corresponding script.In short, we obtained 1 h of Malay speech from this free source.
The training data text script that is specially designed and recorded were obtained from ( Swee, 2014).However, this set of text script was recorded by a male native adult speaker.In short, 1 h of Malay recorded speech was obtained to become training data.
Front end processing using direct mapping letter to sound rule: Unlike conventional speech synthesiz which uses phoneme as the basic synthesis unit, we used letter to be the basic synthesis unit instead.The difference between using phoneme or letter as the training unit is: a dictionary is required to find out the precise phoneme boundary for every ph not required to segment the lexicon into letters.Decode the lexicons into letter is much simpler than in phoneme and requires no knowledge from language experts.Figure 1 shows how the direct mapping letter to sound rule is defined.

Speech training:
The process of training can be categorized into 3 phases.
Phase 1: The features of the original training speech were extracted and variance flooring was applied.Then the Hidden Markov Model (HMM) was initialized using K-mean clustering and re-estimated using Expectation-Maximization (EM) algorithm.After that, the HMMs were converted into context dependent models.
Phase 2: Embedded training of context dependent models without parameter tying was conducted.Then, Res.J. Appl. Sci. Eng. Technol., 7(24): 5143-5147, 2014 5144 (2) speech parameters we want to generate, ӂ is the set of These parameters are then used to generate speech waveform.Any generative model can but HMM is most widely used model in this less ability to reduce complexity during process.It is commonly known as Yoshimura et al., 1999).
The found data for this study is lectures.comwhich is a free resource providing Islamic teaching recording.It offers free download of audio recording of Al-quran reading in Arabic language with translation of Malay segmented the Malay speech portion out and prepared the corresponding script.In short, we obtained 1 h of Malay speech from this free The training data text script that is specially designed and recorded were obtained from (Yong and However, this set of text script was recorded by a male native adult speaker.In short, 1 h of Malay recorded speech was obtained to become

Front end processing using direct mapping letter to
Unlike conventional speech synthesizer which uses phoneme as the basic synthesis unit, we used letter to be the basic synthesis unit instead.The difference between using phoneme or letter as the training unit is: a dictionary is required to find out the ary for every phoneme but it is not required to segment the lexicon into letters.Decode the lexicons into letter is much simpler than in phoneme and requires no knowledge from language experts.
shows how the direct mapping letter to sound The process of training can be The features of the original training speech were extracted and variance flooring was applied.Then the Hidden Markov Model (HMM) was initialized estimated using Maximization (EM) algorithm.After that, the HMMs were converted into context dependent Embedded training of context dependent tying was conducted.Then, Phase 3: Convert trained HMM into HTS models.Viterbi algorithm is then applied to re HMMs.
The training process is illustrated in

Synthesis of speech:
The desired synthetic sentences were formed and labeled like in training stage, resulting in a sequence of context-dependent phone labels for each utterance.Then, acoustic models were joined based on the synthetic sentence.And the speech parameter generation algorithm (Case 1) ( 2000) was adopted to generate the spectral and excitation parameters.The STRAIGHT vocoder is then generates the speech waveform using the parameters.
Evaluation: Five systems (System A to E) have been created to test the reliability of synthetic speech which uses found data as training data.We used the training speech from both recorded data and found data as standard reference.And we designed some normal sentences which is meaningful an Semantically Unpredictable Sentences (SUS) ( et al., 1996) for both recorded data a summary is listed in Table 1.
The SUS design was based on the following structures (Table 2).
Perceptual test was conducted by 17 listeners to evaluate the quality of synthetic speech in terms of naturalness and intelligibility.All the listeners are native Malay speaker.Even though there are some objective methods to test the quality of synthetic speech, but only perceptual test is able to effectively evaluate the naturalness and intelligibility of synthetic speech (Ekpenyong et al., 2014).For naturalness test, listeners were presented the synthetic speeches from all the systems.They were asked to rate the on their opinion about its naturalness mapping letter to sound rule the models were compressed and decision tree clustering was applied.After the models were tied, embedded training was applied again to tied models.And the parameters were untied after the embedded trained HMM into HTS-engine models.Viterbi algorithm is then applied to re-align The training process is illustrated in Fig. 2.
The desired synthetic sentences were formed and labeled like in training stage, resulting dependent phone labels for each utterance.Then, acoustic models were joined based on the synthetic sentence.And the speech parameter generation algorithm (Case 1) (Tokuda et al., 2000) was adopted to generate the spectral and parameters.The STRAIGHT vocoder is then generates the speech waveform using the parameters.
Five systems (System A to E) have been created to test the reliability of synthetic speech which uses found data as training data.We used the original training speech from both recorded data and found data as standard reference.And we designed some normal sentences which is meaningful and intelligible and Sentences (SUS) (Benoît ., 1996) for both recorded data and found data.The The SUS design was based on the following Perceptual test was conducted by 17 listeners to evaluate the quality of synthetic speech in terms of ility.All the listeners are native Malay speaker.Even though there are some objective methods to test the quality of synthetic speech, but only perceptual test is able to effectively evaluate the naturalness and intelligibility of synthetic ., 2014).For naturalness test, listeners were presented the synthetic speeches from all the systems.They were asked to rate the speech based naturalness using a range  from 1 to 5. Five represents very natural while 1 represents least natural.For intelligibility test, listeners were asked to transcribe the perceived synthetic speeches into texts.This listening test was conducted in a quiet room in Universiti Teknologi Malaysia.Headphone was used for every listening test.Each listening test lasts around 40 min as they have to listen to 50 sentences for naturalness test and 50 intelligibility test.

RESULTS
The result of naturalness test is shown in and Fig. 3. Detail Original speech from recorded data and found data Synthetic speech from recorded data using normal sentences Synthetic speech from found data using normal sentences Synthetic speech from recorded data using SUS Synthetic speech from found data using SUS represents very natural while 1 represents least natural.For intelligibility test, listeners were asked to transcribe the perceived synthetic speeches into texts.This listening test was conducted in a quiet room in Universiti Teknologi Malaysia.
ne was used for every listening test.Each listening test lasts around 40 min as they have to listen test and 50 sentences for The result of naturalness test is shown in Table 3 Listeners were asked to transcribe the sentences into text.From the response of listeners in this test, we calculated the Word Error Rate (WER) according to the equation below: where, S is substitution of words words, I is insertion of words and Table 4 shows the Word Error Rate (WER) of all systems.

DISCUSSION
Using both recorded data and found data as training data, the naturalness of synthetic Original speech from recorded data and found data Synthetic speech from recorded data using normal sentences Synthetic speech from found data using normal sentences from recorded data using SUS Synthetic speech from found data using SUS Kangkung ini bersambilan dengan pendengaran yang besar.
Manakah orang itu menolak lampu yang bising?E 3.5612±0.2552 Listeners were asked to transcribe the sentences into text.From the response of listeners in this test, we calculated the Word Error Rate (WER) according to the  Listeners may find it unnatural since it is not intelligible and meaningful in terms of sentence content.On the other hand, similar trend happened in intelligibility test.The WERs of normal sentences is close to the WER of original speech.And the WER of SUS is similar for both speeches trained by recorded data and found data.However, there is a noticeable increase in WER of SUS compared to normal sentences.It may due to the random placement of words in the SUS due to the nature of SUS so listeners were feeling difficult to perceive the correct words.
In this study, the naturalness and intelligibility of synthetic speech trained by found data is satisfactory and listeners were able to perceive the meaning of normal sentences.This is a great ease of training data collection process because recording database and constructing recording script is tremendous and requires good quality of recording setup.However, there are a lot of free source like educational audio-book, storytelling book, speech and so on can be found online.The quality of the recording of the free source can be good enough to be the training data.Manually segmentation can be done to select only clean and clear speech to be the input data.

CONCLUSION
We have presented a Malay language speech synthesizer in this study.We compared the synthetic speech trained by recorded data and found data.Recorded data were obtained from a series of procedure from words collection, sentence design and recording under good quality of recording setup while the found data was obtained from free source like audio-book, speech and so on.The listening test result showed no significant difference between synthetic speeches trained by recorded data and found data.It is an encouraging result to show that alternative source of training data is able to become training data while a lot of human efforts were bypassed in preparing the training data.
To mention future work, different accent of free speech source can be used to synthesize speeches in different accent.Automatic segmentation of clean speech like diarization of speech can be conducted to reduce more human effort.

Fig. 2 :
Fig. 2: Block diagram of training process D is deletion of and C is correct words.shows the Word Error Rate (WER) of all data and found data as synthetic speech of

Fig. 3 :
Fig. 3: Result of naturalness testnormal sentences is close to the original recorded speech.And the synthetic speech of SUS is slightly lower than normal sentences.But the naturalness is similar for both synthetic speeches using recorded and found data.The slightly decrease of naturalness in SUS may due to the understanding of the sentences.Listeners may find it unnatural since it is not intelligible and meaningful in terms of sentence content.On the other hand, similar trend happened in intelligibility test.The WERs of normal sentences is close to the WER of original speech.And the WER of SUS is similar for both speeches trained by recorded data and found data.However, there is a noticeable increase in WER of SUS compared to normal sentences.It may due to the random placement of words in the SUS due to the nature of SUS so listeners were feeling difficult to perceive the correct words.In this study, the naturalness and intelligibility of synthetic speech trained by found data is satisfactory and listeners were able to perceive the meaning of normal sentences.This is a great ease of training data collection process because recording database and constructing recording script is tremendous and requires good quality of recording setup.However, there are a lot of free source like educational audio-book, storytelling book, speech and so on can be found online.The quality of the recording of the free source can be good enough to be the training data.Manually segmentation can be done to select only clean and clear speech to be the input data.

Table 1 :
Systems created for listening test

Table 4 :
Word Error Rate (WER) of each system