Main Principles and Overall Structure of TTS system for Russian Language

Olga Krivnova

Philological Faculty, Lomonosov Moscow State University, Russia

Abstract: The paper describes the main principles of Russian text-to-speech synthesis developing by Speech group of the Philological Faculty, Lomonosov Moscow University, Russia. The system is organized as a mixture of two methods: concatenation - on the segment level (using the linguistically motivated units - allophones' waveforms spliced together to form the initial speech wave) and the rule-based method on the prosodic level to modify the initial speech wave according to the prosodic characteristics of a phrase being synthesized. 

OVERALL ARCHITECTURE OF THE SYSTEM
Our system consists of the following functional blocks (or modules): Text normalization module; Two Automatic Transcribers Modules: Accent-Intonation and Phonemic; Allophonic Coding Module; Block of Prosodic Parameterization; Allophonic Data-Base (acoustic inventory); Control File Generation Module; Speech Signal Generation Module. The overall architecture is in line with the functional organization of a general TTS synthesizer.
Our system is designed as a research prototype, at the same time we wanted to build such a system that could be easily edited, tested and changed. Therefore special attention in our work is given to development of special toolkit, which is used for operative testing of results of both: all system as a whole and its separate modules.
Besides the system is used in the educational purposes: Within the framework of cources on Russian phonetics and speech technologies. 
	Synthesizer is realized in library DLL for platform Win32 (Windows 95 or NT); the place occupied on a disk about 10Mb (without wave compression). 
	The synthesis of an arbitrary sentence may be split into two larger stages. On the first stage the task is to produce a phonetic (symbolic) transcription of the text to be pronounced including its intentional, accentuation and rhythmical characteristics. On the second stage the necessary acoustic signal (in digital form) should be created on the base of this phonetic representation. These tasks are implemented in our system by text preprocessing module and digital signal processing module accordingly. These blocks are connected through a module termed as speech control generation which function is to generate a list of allophone codes (names) augmented with necessary prosodic information (frequency, duration, energy).
	The overall structure of our synthesizer is shown on Fig.1. The more detailed description of system's modules will be given below (see also (1)). 

TEXT PREPROCESSING
Input text normalization. The main task of text preprocessing stage is the construction of phonetic representation of an input text and its fragments, which should contain segmental and suprasegmental information necessary for the subsequent generation of an acoustic signal. 
For creation of such transcription the input text should be at first converted to the sequence of the orthographical words with stress marks; the words must be divided by white spaces and 

					FIGURE 1. Overall structure of Russian TTS system. 

Fig.1. Overall structure of TTS system 

admissible punctuation marks. Compound words should have more than one stress mark the last of which is taken as primary. Such text form can be conventionally named "normalized".
The operations, which are necessary to transform the input text into the normalized form, concern to a stage of text normalization, which we understand wider than it is accepted in many TTS systems. We differ the following tasks here :
•	Processing textual objects, which are not ordinary words (numerals, abbreviations of a type "Mr., etc., i. e." etc.) and require verbal expansion with a choice of the necessary grammatical form.
•	Processing textual objects, which are not ordinary words, but at the same time do not require verbalization (acronyms of a type "MSU" , proper names). As a rule such words are absent in the ordinary dictionaries.
•	Processing ordinary words (stress marker placement, replacement the letter "е" on the letter "ë" in those cases where it is needed for correct word pronunciation, as in Russian texts the letter "ё" practically is never used. 

Regretfully, we did not elaborate systematically the first two tasks. Nevertheless we have collected a number of preparatory materials for their development: on a material of the newspaper texts a list of acronyms with their frequency of occurrence, phonetic transcriptions and word collocations – sources is made (electronic version 200 entries) and also a list of abbreviations (with period in spelling) used in scientific texts with their verbal expansion in dictionary canonical form (electronic version 1100 entries). Recently we have developed rules to process alphabetic acronyms. Besides we can process numbers in expressions of dates and time of the day. But as a whole we think that many problems arising while implementing the tasks 1 and 2, demand the contextual analysis with use of at least grammatical information. To receive this information it is necessary to perform morphological analysis of words processed and the same is true for the stress assignment to Russian words.
As it is well known, Russian has a so-called free stress with strongly reduced unstressed vowels. Pronunciation errors connected to stress misplacement are very appreciable on hearing and frequently make a word practically unrecognizable, let alone that in a number of cases the stress placement has distinctive function (as in /mu'ka/ "torment" vs /muka'/ "flour" or /ru'ki/ vs /ruk'i/ as different grammatical forms of a word "hand"). So in Russian the place of a stress is lexically and grammatically conditioned.
Full morphological analysis of ordinary Russian words is not an easy problem. Fortunately, there exists (electronic version also available) the grammatical dictionary created by A.A.Zaliznjak (2), which contains about 100000 words in their initial forms and all information necessary for the analysis / synthesis of their grammatical forms. The most of morphological processors developed for Russian use this dictionary by and large and we including. 
As against the majority of existing morphological processors for Russian in our analyzer the theoretical model of word inflection described in (2) is used practically without changes. It was achieved by using the well known principles: calculations of stems, analysis by synthesis, statement and verification of hypotheses by various modules (principle of "a class board "). 	
In (2) all possible accentuation word types are accounted for and all initial word forms have markers for stress position and position of the letter "e" to be replaced by letter "ë". Also for each word entry (2) specifies: part-of-speech, gender and animacy (for nouns), aspect and transitivity (for verbs), type of inflection paradigm. Due to the principle "analysis-by-synthesis" this information allows not only to assign stresses, replace "e" by "ë" (what is performed in our system), but also to receive all grammatical features possible in case of an isolated word treatment. We are going to use these features for contextual analysis which is necessary for different tasks: unusual symbols to word expansion, homographs and other types of ambiguities, prosodic phrasing etc. Now it is only in a stage of development. Manual stress placement and creation of a special user dictionary is provided in our system also. 
AUTOMATIC ACCENT-INTONATION TRANSCRIPTION (AITR)
Prosodic word grouping and phrasing. Within	 a 	phrase	 this process 	is 	closely 	connected with rhythmization of word sequence and is realized in our system by special feature "degree of prosodic break", assigned to the blanks between words. Here we have three breaks' levels: 0 - between full clitic (e.g. preposition) and content word; 1 - between functional word and meaningful word; 2 - between meaningful words. This information is accounted for in phonological rules when processing external phoneme sandhi and vowel reduction. Major cues to locate prosodic boundaries are punctuation marks. However it is well known that the correspondence between intonation and punctuation is not direct one. Besides, the information about major syntactic breaks and the length of the main constituents are at least needed. Now we conduct work in this direction, however while the intonational phrasing in our system is based exclusively on punctuation. It is possible also to indicate phrase boundaries in the input text manually.
Prosodic phrasing is supposed to have some pause after each phrase. We distinguish three degrees of pauses: short (about 250ms), moderate (about 400 ms) and long (about 800 ms). As far as the localization of phrase boundaries is concerned this problem is under investigation now and at present their places are fully determined by punctuation marks. 
Intonation. To each phrase AITR assigns one of 7 intonation models. In our system we use the following models: 2 models of finality; 2 models of non-finality for the affirmative sentences, 3 interrogative models (general question, special question, comparative question), 1 model for exclamation sentences. For all models the possibility of different positions of melodic center in the phrase is available (in some cases its position is determined by the nuclear phrase stress, while in others by focus accent mark). The choice of the appropriate model is based also on punctuation marks and some lexical information. It is obvious that these cues are not enough and more over the relation between punctuation and intonation models is rather difficult one, especially in Russian . This problem is also under investigation. At a present stage of development any phrase can be pronounced with any of accessible intonational models in a hand-operated mode.
Rhythm and accentuation. AITR assigns some degree of prominence to each vowel in the synthesized phrase as its rhythmical feature. We distinguish three degrees of syllable prominence within a word, four degrees of lexically stressed syllable prominence within a phrase ( 1 for full clitics, 2 for functional words, 3 for nonnuclear meaningful words, 4 for nuclear meaningful word). The last meaningful word in the phrase is considered to be nuclear phrase stress by default. Though we are able to synthesize phrases with different focus accents we have no rules to determine their localization automatically: it should be done manually.
AUTOMATIC PHONEMIC TRANSCRIPTION (PHTR)
The phonemes inventory includes 56 units. It slightly differs from that prevalent in Russian phonetic descriptions. In some cases it is convenient to have different symbols in the transcription even for those phone pairs that are in no meaningful contrast. 
Standard phonological rules of PHTR implement the mapping "letter - phoneme" and "phoneme - phoneme" which include such operations as elimination of spelling fictions, processing of pronunciation for some consonant clusters, removing of hard and soft signs in spelling and so on. The processes working as within a word and between words are taken into account. Irregular pronunciation of some word classes (e.g. loan words) and even individual words is accounted for by using special word lists. Currently there are 54 such lists in PHTR.

PROSODIC PARAMETRIZATION
Duration. The duration rules were designed for and are applied separately to vowels and consonants. The duration patterns of vowels are formed in accordance with their prominence levels, phonetic quality, boundary phrase position and some other factors. The general rules for consonant duration are also based on several factors: position regarding the phrase boundaries; intervocal - non-intervocal position; position in the consonant cluster; the prominence level of the following vowel and so on. In our system it is possible to control the overall tempo of pronunciation (evenly, only on consonants or only on vowels). 
Pitch and fundamental frequency. The melodic patterns modeling is based on generating tone turning points (or target points of tone inflection) and their parameters in frequency and time domains. The rules for phrase melodic patterns assign usually two tonal values (in semi-tones) to every allophone - as its starting and final points. If it is necessary the third value can be assigned to any point inside the allophone. So, on the whole any melodic contour is approximated by linear tonal movements. Melodic contour described in semi-tones is then transformed into Hz values accounting voiced - un voiced feature of the consonants. This procedure takes also into account the Hz conformity of the base tone characteristic for a speaker. 
Amplitude. Basically those characteristics are kept which are inherent to allophones as the concatenation units, i.e. in most cases no updating of the initial data occurs. We use some rules to control the changes of amplitude on the final syllables in a phrase and some rules of adjusting type. 				
ALLOPHONIC CODING MODULE
This module converts the transcribed phrase into the codes (names) of appropriate concatenation units. As a rule these units are phoneme-size acoustic segments (i.e. allophones) and their codes contain just the information on their phonemic identity and on the classes of their right and left phoneme contexts. There are two exceptions from this trend, which require some additional processing. So, stops, affricates and trills can be spliced on several acoustic segments and on the other hand some two-phoneme sequences (i.e. vowel hiatus) can be merged into a single acoustic segment. 
GENERATION OF SPEECH CONTROL FILE
The outputs of prosodic parameterization block and allophonic coding module are united in a general control file where a synthesized phrase is represented as a sequence of concatenation unit codes with assigned duration, F0 and amplitude values.

								SPEECH SIGNAL GENERATION
Signal generation is implemented according to the phrase control file the structure of which was described above. The necessary allophones are extracted from the database and spliced together. To transform the base allophones to physical values given by the phrase control file, we use procedures that are close to PSOLA technique in the time domain. To increase the naturalness of synthesized speech at last stage of a signal generation the final sequence of the modified pitch periods is transformed from linear approximated curve into cosine form and some casual fluctuation of period duration's is implemented.
ALLOPHONIC DATA-BASE
The allophonic database is a set of allophone wav files, each file being named accounting the allophone itself and its phonetic context. We use two acoustic inventories: one for male voice (SR 11025Hz; SS 8-bit) with 158 consonant allophones and 530 vowels; the other for female voice (SR 22050Hz; SS 16-bit) with 200 consonant allophones and around 1100 vowels. All vocal sounds in the data-bases are marked semi-automatically according to their pitch periods to generate the output speech signal. The main principles of the database design and its preparing was described in (3). The basic requirement was to cover possibly all acoustically and perceptually relevant effects of coarticulation, not exceeding at the same time the reasonable bounds of database size.

REFERENCES
1.	Krivnova, O.F., "TTS synthesis for Russian language (second version for female voice)". Proceedings of the International Workshop on Computational Linguistics and its Applications "Dialogue98". Kazan, 1998. (In Russian).
2.	Zaliznjak, A.A., Grammatical Dictionary of the Russian Language. Moscow: Russkij Yazyk, 1977.
3. 	Zinovieva, N.V., "Phonetically sufficient allophonic database for concatenation synthesis of Russian speech". Proceedings of the XIIIth ICPhS, Stockholm, Sweden, 1995.