Speech propagates as a longitudinal wave in a medium, such as air or water. The speed of propagation depends on the density of the medium. It is common to plot the amplitude of air pressure variation corresponding to a speech signal as a function of time; this kind of plot is known as a speech pressure waveform or just a speech waveform. Here's an example taken from part of a vowel:

The plot shows the change in air pressure with time and we can see that in this example the pattern of changes is regular in that each up-down cycle has roughly the same shape. This corresponds to a periodic signal and these signals make up the voiced speech sounds such as vowels, and voiced consonants such as `b', `w' and `z'. Another kind of sound is irregular or aperiodic and so appears to be just random variations in air pressure. Examples are the hissing sound in `s' and `sh'.

The simplest form of sound that we can describe is called a sinusoid. This waveform corresponds to a pure tone and as we will see later forms the basis for all more complex sounds. Since a sinusoid has such a pure shape it is a useful starting point when describing the properties of sound waves.

A sinusoid corresponds to a simple smooth up and down movement when drawn as a time waveform. Three kinds of measurement can be taken from a sinusoid and together they completely define the shape of waveform. These measurements are:

When two sinusoids are added together the result depends upon their amplitude, frequency and phase. The effects are easiest to observe when only one of these is varied between the two sinusoids being added.

In the simplest case, when two sinusoids with the same frequency and phase but with different amplitudes are added together the result is a sinusoid who's amplitude is the sum of the originals and who's frequency and phase remain unchanged.

When the two sinusoids have different frequencies the result is more complicated. The new signal is no longer a sinusoid since it doesn't follow the simple up and down pattern. Instead we see the higher frequency sinusoid as a `ripple' superimposed on the lower frequency sinusoid (see figure). The frequency of the resulting signal will be the lower of the two original frequencies. In the figure we can see that the resulting signal goes through two and a half cycles (that is it repeats itself this many times) just like the second sinusoid. The amplitude of the resulting signal is the sum of the originals and the phase is unchanged.

Adding together sinusoids with different phases can have surprising results. If two sinusoids are `in phase' then their peaks and troughs coincide and the result is as observed earlier. If the sinusoids are `out of phase' or in other words if they differ in phase by π radians then their peaks and troughs oppose each other and they will cancel each other out. The phase of the resulting sinusoid is the sum of the phases of the constituents.

Adding up more than two sinusoids can produce complex looking waveforms. When a number of different frequencies are combined the frequency of the result will in general be that of the lowest frequency component. This is often called the fundemental frequency of the signal.

Fourier theory says that any complex periodic waveform can be decomposed into a set of sinusoids with different amplitudes, frequencies and phases. The process of doing this is called Fourier Analysis and the result is a set of amplitudes, phasesand frequencies for each of the sinusoids that makes up the complex waveform. Adding these sinusoids together again will reproduce exactly the original waveform.

A plot of the frequency or phase of a sinusoid against amplitude is called a spectrum. For a pure sinusoidal tone this is just a single line as shown in Figure 2.9. of Harrington and Cassidy. An amplitude spectrum of a complex waveform shows the amplitude of each frequency component or sinusoid. The phase spectrum shows the phases of these components but is very seldom used in speech analysis. Spectra are used extensively in speech analysis to discern the characteristic shapes associated with different classes of speech sound.

Any periodic signal shows a pattern of repitition in the time waveform which corresponds to the primary rate of vibration of the signal, known as the fundamental frequency. This corresponds to the lowest major frequency component in the signal and, in the case of voiced speech, equates to the frequency of vibration of the vocal folds. A vibration source like the vocal folds also produces what are called harmonics which are oscillations at multiples of the fundamental frequency. For example, a 100Hz source vibration will give rise to harmonics at 200Hz, 300Hz, 400Hz etc. In the spectrum of such a signal, the harmonics show up as spikes at these frequencies.

The fundamental frequency can be measured from a speech waveform by looking for the period of oscillation of the signal around the zero axis. Estimates can also be made from the spectrum since it shows a large peak at this frequency and at each multiple due to the harmonics. Measurements can be made of the frequency of the major peak or of the distance between harmonic peaks.

A spectrum shows the freqency content of a short section of a waveform but it we want to see how the spectrum changes with time through a section of speech we need a different kind of display. A standard way of displaying this information is a spectrogram; this is a two dimensional plot of frequency against time where the amplitude at each frequency is represented by the darkness of the corresponding point in the display. The figure shows an example spectrogram aligned with the speech waveform for the word `stamp'. Experienced phoneticians can glean a lot of information from these kinds of displays and you will look at lots of them during your later practical work. For now we can observe a few prominent features: the dark are corresponding to the high frequency energy in the /s/ phoneme, the vertical striations in the /A/ phoneme corresponding to the fact that this vowel is a periodic signal (voiced) and the sudden onset of the relatively light area corresponding to the release of the /p/ at the end of the word.

The vertical striations in the vowel of `stamp' give another way of measuring the fundamental frequency of the signal at this point. In this kind of spectrogram (called a wideband spectrogram) each vertical striation corresponds to one open-close cycle of the glottis; hence measuring the time between striations can tell us the period of oscillation and hence the fundemental frequency of the speech signal at this time.

The worksheets included with Harrington and Cassidy contain a number of exercises which are useful in illustrating the properties of sinusoids and spectra. Section 2 of the 'Interactive Workseets' illustrates the shape of sinusoids and the result of adding them together, measurements of frequency from complex waveforms and shows some examples of spectrograms.

Basic acoustics is covered in both Ladafoged (Chapter 8) and Harrington and Cassidy (Chapter 2). For a much more detailed and slower paced discussion of sound propagation and the properties of waveforms see Introduction to Sound by Charles E. Speaks published by Singular Publishing (ISBN: 1879105985).

Much of this unit will be concerned with the manipulation of speech signals by digital computers. The power of modern desktop computers means that advanced signal processing and speech analysis software can run on the average computer. In order to take advantage of this, and in order to understand the properties and limitations of digital signals, we need to look at how a speech siganl is converted into digital form.

Once a digital signal has been captured it can be stored inside the computer as a sequence of numbers. Our aim in doing this is to be able to manipulate and analyse this digital form to extract information from it and convert it to different forms. To this end we need to look at the concept of vectors and the operations of vector arithmetic.

A vector is the mathematical name for a sequence of numbers of arbitrary length. In Harrington and Cassidy and in these notes we write a vector as a list within square brackets:

[1, 3, 19, 299, 34, 12]

x = [1, 3, 19, 299, 34, 12]

A number of simple operations are possible on such vectors and these are described in Harrington and Cassidy Section 5.3, page 138. In summary we can add a scalar (a scalar is a single number, as opposed to a vector which is a sequence of numbers) to a vector, multiply by a scalar or add/multiply two vectors. When combining vectors in this way both must have the same length since the elements are combined in a pairwise manner. See the text for examples and notation.

When a vector is used to denote an acoustic signal some more assumptions are made. Acoustic signals are notionally infinite (they have no start or end) so for any finite vector we assume that the numbers before and after the vector exist and are zero. This becomes important when we shift vectors to the left or right (a useful operation in some common signal processing operations) -- in this case a zero is shifted in the the first (or last) position of the vector.

The exercises for this section come from the "Practical Exercises with XlispStat" -- the second set of worksheets on the CDROM accompanying Harrington and Cassidy. These worksheets require that you learn a small amount of the Lisp language although most of the exercises can be completed by copying from the examples in the worksheets. You should begin with the introduction in section 2 which covers the basics of the language and environment. The exercises for later chapters will build on the basic commands that you learn here.

For this chapter you should work through the worksheets in section 3 (Time Domain Processing) up to section 3.4. Make sure that you are comfortable with the notion of storing sequences of numbers as vectors and the operations that you can perform on them. You will use these operations later to perform some simple digital signal processing on speech signals.

This material is covered in Harrington and Cassidy, Chapter 5, sections 5.1 - 5.3.

When we analyse signals we tend to do so on only small portions at a time. The reason for this is that we typically assume that the signal is constant (eg. has a constant frequency) over the time-span of our analysis. Since speech is actually changing rapidly, we have to cut it into small parts for this assumption to hold. Hence we typically window a speech signal before analysis, which just means that we cut out a small section.

Windowing can be seen as multiplying a signal by a window which is zero everywhere except for the region of interest, where it is one. Since we pretend that our signals are infinite, we can discard all of the resulting zeros and concentrate on just the windowed portion of the signal.

The above window (zeros and ones) is known as a rectangular window because of its shape. One problem with this kind of window is the abrupt change at the edge, which can cause distortion in the signal being analysed (in fact any windowing operation causes distortion, since the signal is being modified by the window). To reduce this distortion we often use a smoother window shape called a Hamming window. This window is zero at the edges and rises gradually to be 1 in the middle. When this window is used the edges of the signal are de-emphasised and the edge effects are reduced.

It is important to use a Hamming (or the similar Hann window) in some kinds of analysis, particularly the frequency domain methods we will look at later. For many time domain methods the rectangular window is preferable.

If we are analysing a long signal, then successive windows are taken along the length of the signal. Often we overlap the windows if a Hamming window is being used so that the de-emphasised part of one window becomes the middle of the next.

You should again do some exercises from the `Practical Exercises...' worksheets from Harrington and Cassidy. Continue with section 3.5 on windowing signals and then look at the first part of section 4 on speech signal parameters which gives examples of calculating autocorrelation coefficients for a sample signal. Leave the remaining exercises in section 4 until later.

(def w (send stamp :window :start 2000 :width 1024 :type 'rectangular)
(def w-ac (send w :autocorr))
(send w-ac :plot)    
      

(window-demo stamp :transform :autocorr :width 1024 :type 'rectangular)
      

Chapter 5. of Harrington and Cassidy, sections 5.4 - 5.6

The Fourier transform transforms a time domain signal into a frequency domain representation of that signal. This means that it generates a description of the distribution of the energy in the signal as a function of frequency. This is normally displayed as a plot of frequency (x-axis) against amplitude (y-axis) called a spectrum.

In digital signal processing the Fourier transform is almost always performed using an algorithm called the Fast Fourier Transform or FFT. This is, as its name suggests, a quick way of performing this transform and it gives the same results as the slower Discrete Fourier Transform (DFT) would. We needn't worry here about any of the details of how the transform is carried out, we are interested only in the results. One consequence of using the FFT algorithm is that the length of the signal being analysed must be a power of two, eg. 128, 256, 512, 1024 and so on (although many modern implementations get around this limitation).

Applying the FFT algorithm to a 512 point window of data taken from a speech signal results in a vector of 512 numbers corresponding to the energy at 512 frequencies spanning the range from 0Hz to the sampling frequency. From Fourier's theorem we know that adding together 512 sinusoids at these frequencies will exactly reconstruct the original signal. From Nyquist's theorem we know that the largest frequency component in the original signal must be half the sampling frequency, all frequencies above this are aliased onto lower frequencies. It follows that the second half of the spectrum is the mirror image of the first half:

In fact, the first point in the spectrum isn't reflected and the frequency range from 0 to half the sampling frequency is covered by (N/2)+1 points where N is the size of the window. So from a 512 point fft of speech sampled at 10000Hz we get 257 unique spectral points covering the range 0 to 5000Hz.

For any digital spectrum we can work out the space between points in on the frequency axes, in the example above this will be 5000/257 or 19.5 Hz. This means that if there are two peaks (or dips) in the spectrum separated by less than 19.5Hz we won't be able to resolve them -- they will be blured together. If we used a wider window, say 1024 points, the space between points on the frequency axis drops to 9.7Hz, now we can see more detail than before. The resolving power of a spectrum is called it's bandwidth, refering to the width of the frequency bands that each point in frequency represents. A large bandwidth implies few points in the spectrum and hence poor resolution in frequency; a small bandwidth implies many points in the spectrum and good resolution in frequency.

There is a tradeoff between resolution in frequency and resolution in time. To get good frequency resolution we need to take a large window of points from the signal. By analysing this large window we blur any spectral changes which happen over it's length -- that is, if two events occur close together in time, a large window will tend to blur them together. Hence good frequency resolution implies poor time resolution and good time resolution implies poor frequency resolution. We can see the result of this tradeoff in wide and narrowband spectrograms (end of Chapter 2 in the text). The wideband spectrogram has a large bandwidth (small window) and so blurs the frequency scale but gives good time resolution -- you can see vertical striations on the spectrogram corresponding to the opening and closing of the glottis. The narrowband spectrogram has a small bandwidth (large window) and so looses time resolution but displays excellent frequency resolution -- you can see horizontal bands corresponding to the harmonics of the pitch frequency.

The figure below shows dB spectra corresponding to a section of the vowel in stamp calculated withh window sizes of 1024 points (red) and 256 points (black). Notice the peaks due to pitch harmonics in the red line which are blurred over in the black.

When the window size is very small, the spectrum generated by the fourier transform will have a correspondingly small number of points and hence will appear very jagged. In order to produce a smoother result in this case the input window can be zero-padded to extend the size of the window without taking more of the input signal. The fourier transform of the zero padded signal will have many more points and the result is that the spectrum appears to have been smoothed out. An example can be seen in Figure 6.1 which shows a raw spectrum taken on 50 points next to a smoother version taken from the same 50 points padded with 500 zeros.

We commonly transform a spectrum as calculated by the fft into a decibel spectrum to better reflect the response of the human ear. The units of amplitude in the raw spectrum shown above are the same as those of the sampled acoustic signal. The decibel scale is a logarithmic scale of relative amplitude where we first devide by a reference amplitude (which corresponds to 0dB) and then take the naturnal logarithm. Finaly we multiply by 20:

AdB = 20*log(A / Aref)

where AdB is the amplitude in decibels, A is the raw amplitude and Aref is the (raw) reference amplitude. There are five basic choices for Aref:

The choice of value for Aref depends on the needs of your application. The big question is whether you want to preserve differences in amplitude between spectra or between utterances. For example, if you think that amplitude is going to be a useful cue then you need to preserve the relative amplitude of your spectra, so you can't use either method 3 or 4 above. If you want to factor out the overall loudness of a recording then method 2 might be useful. If you want to get precise relative amplitude measures then you need to use a calibrated system (method 1).

Filtering is an operation on an acoustic signal that changes the shape of the spectrum of the signal. A filter is a device (or an computer program) which applies such a change to a signal. Examples of filters are the bass/treble controls on your stereo which attenuate the higher or lower frequency ranges in the music signal. Each filter has a characteristic spectral shape which describes which parts of the original signal will be attenuated and which will be left alone. An example for a low pass filter might be as shown in Figure 6.2.

The effect of applying a filter to a signal is to modify the shape of the signal's spectrum. In the frequency domain the effect of applying a filter to a signal is to multiply the spectrum of the signal by that of the filter. (Note that we multiply the raw spectra and not the dB spectra -- since dB spectra are logarithms they should be added to achieve the same effect since log(a*b) = log(a)+log(b).) The result is a spectrum that combines the features of those of the input signal and the filter. An example is shown in Figure 6.3 (taken from Harrington and Cassidy, p. 191) of a simple impulse train input signal (which has a `flat' spectrum) filtered through a bandpass filter; the output signal shows the characteristics of both spectra.

Having applied a filter by combining the spectra, a new filtered signal can be derived by performing an inverse fourier transform on the new spectrum.

The question arises as to how we can find the spectrum of a filter, or indeed how a filter is represented digitally. A digital filter can be characterised by an acoustic signal called an impulse response -- literally this is the result of passing an impulse (a 1 followed by lots of zeros) through the filter. The spectrum of a filter can be derived by taking the spectrum of its impulse response.

Filters are an incredibly important topic in the study of speech acoustics. As we will see, the vocal tract acts as a filter which changes the spectral characteristics of the acoustic source generated by the glottis or by frication. Understanding the nature of filters and the separability of the source and filter are fundamental to the study of speech acoustics and form the basis of much of speech technology research.

Although it is possible to apply a filter in the frequency domain by multiplying raw spectra, this is not the most practical way of applying a digital filter. Fortunately there is a time-domain operation that has the equivalent effect: convolution. Convolution is an operation that combines two vectors to give a third vector. If the first vector is an acoustic signal and the second is the impulse response of a filter, then the result of convolution is a filtered signal.

The process of convolution is described in detail in Harrington and Cassidy, Section 5.6, p. 148. It involves a series of multiplications and additions of corresponding points in the two vectors. Rather than reproducing the explanation here I refer you to that discussion.

An important point to take away from the discussion of convolution is that the operation is equivalent to weighted differencing of the input signal. The filter provides the weighting coefficients. Also, remember the formula y = x*h where y is the output signal, x is the input signal and h is the filter impulse response. We will see this formula later when we look at more advanced DSP methods; it provides an important key to analysing speech signals.

For this section you should complete some of the worksheets on the source filter model from the `Interactive Worksheets on Speech Acoustics' on the Harrington and Cassidy CDROM. Worksheet sections 3.1, 3.2 and 3.3 are relevant. The later sections will be useful when we talk about the source filter model of speech production in the next section.

There are two acoustic sources in speech production corresponding to voiced and unvoiced speech sounds.

We have seen that a filter is a system that alters the frequency content of an acoustic signal. The filter in speech production is the vocal tract, a tube of uneven cross section which is closed at one end (by the glottis) and measures around 17.5 cm for the average male. As with any other filter, this tube has a characteristic spectrum. Importantly, this spectrum changes as the shape of the vocal tract changes during speech production. The different qualities of the speech sounds are produced by changing the shape of the vocal tract to produce a particular set of filter characteristics.

To understand the effects of the vocal tract we can model it as a cylindrical tube, closed at one end, which makes the calculation of the filter spectrum much easier. The spectrum of this tube has a set of peaks which correspond to resonant frequencies -- basically the frequencies of vibration which fit into the tube best. For a full discussion of this model see Harrington and Cassidy Section 3.3.1. Resonance is the property which allows you to get a single note out of a half full beer bottle by blowing over the top: the wavelength of the vibration is such that it fits within the tube. Change the length of the tube (by taking a swig of beer) and the note changes because now a different wavelength fits the tube best. In fact you can get more than one note if you are well practiced -- this is how a trumpet can play so many notes with only three `keys'. Each length of tube can fit a number of different resonances but the resonances frequencies are all related to the length of the tube.

The end result then, is that a cylindrical tube has a spectrum with a set of peaks at around 500Hz, 1500Hz, 2500Hz etc. which correspond to the resonant frequencies of the tube. Now, if the shape of the tube is changed, for example by adding a constriction halfway along it, the position of these resonances changes also. In the case of a constriction, the effect is to move the resonant frequencies relative to each other -- so that they are no longer equally spaced. Translating this back to the real vocal tract, the cylindrical tube corresponds approximately to a relaxed vocal tract and the corresponding pattern of resonance matches that of the schwa vowel. Constrictions are introduced by raising the tongue and moving the position of the raised tongue back and forward. This has the effect of modifying the positions of the resonances relative to each other, producing the different vowel sounds.

Another filter is involved in speech production, corresponding to the effect of the lips and the air beyond them on the acoustic signal. This lip-radiation filter has a characteristic spectrum which is a 6dB/octave rise -- that is the lower frequencies are attenuated wheras higher frequencies are not. This spectrum is multiplied by the spectrum of the vocal tract to produce an overall spectrum for the system.

As discussed earlier, the effect of filtering a waveform can be achieved either by convoltion in the time domain or by multiplication in the frequency domain. To produce speech sounds the source (voiced or unvoiced), is passed through the vocal tract filter as described above.

For voiced speech, the combined signal has peaks due to the harmonics of the glottal source superimposed upon peaks due to the resonances of the vocal tract. These peaks of the combined spectrum are formants, which as was discussed earlier in the unit are characteristic features of voiced speech sounds. It is important to note that the formant peak in the resulting spectrum will occur on the closest source harmonic peak to the resonance peak -- hence the resonant frequency and the formant frequency aren't necessarily the same thing.

Fricative speech sounds do not display the harmonic peaks of voiced speech but the characteristic peaks of the vocal tract filter can still be seen. Again, the position of these peaks serves to characterise the speech sound since it is intimately related to the vocal tract configuration that produced it.

In the last section we discussed a simplified model of speech production which used a single tube of constant cross section to model the vocal tract. As was mentioned their, a slightly more realistic model includes a constriction in the tube to represent the position of the tongue during articulation. Fairly detailed models of articulation can be developed by using a small number of tubes to model the vocal tract -- a common model uses one tube for the area behind the tongue, one for the constriction created by the tongue, one for the area front of the tongue and a final tube for the lip apperture. The lengths and cross sectional areas of these tubes can be adjusted to account for a number of speech sounds. This model is very useful in predicting the gross changes of formant positions caused as the tongue is raised or lowered and moved from the back to the front of the mouth.

As discussed in Chapter 3 of Harrington and Cassidy, the effects of different vocal tract configurations can be explored via a nomogram which is a plot of the predicted resonant frequencies in a four tube model as the constriction tube is moved from along the vocal tract. The tube configuration is illustrated in Figure 7.2 while Figure 7.3 shows an example nomogram.

You can explore tube models in the exercises for this lecture. These models illustrates one approach to generating synthetic speech, namely articulatory synthesis. Models with a larger number of tubes can be used to produce more accurate reproduction of various speech sounds. It is also possible to derive a tube model for a given speech sound via LPC analysis -- this won't be covered in this unit but is discussed in Harrington and Cassidy, Chapter 8. Such a derived tube model can be used to re-synthesise speech by applying a suitable source to the model.

The source filter model describes the process of speech production in terms of two independent contributions from the sound source and the vocal tract filter. As a model of production, this allows us to synthesise sounds as described in the previous section, but the model can also be used to aid the analysis of speech sounds. The key is to notice the importance of the independence of the source and filter; this means that each makes a separate contribution to the characteristic features of the resulting speech sounds. The source, in voiced speech, is responsible for the pitch of the sound while the vocal tract filter is responsible for the location of the formants and the overal spectral shape. In a real speech spectrum the overall filter shape and the location of the formants is often drowned out by the effects of the source spectrum. If it were possible to remove the source effects then the two spectra could be studied separately to give a more accurate picture of the precursors of the speech sound.

Removing the effects of the source from a spectrum also has the effect of removing many sources of variability from the signal since many of these are realised in the source rather than in the filter. Hence removing the source can help make, for example, the vowel sounds more distinct from one another and could make vowel sounds from different speakers look more similar.

There are a number of techniques for separating source and filter in a speech signal. In this section I will discuss only one of these: Cepstral analysis.

Cepstral analysis relies on the observation that a logarithmic speech spectrum is made up from the source and filter spectra added together. To understand this, recall that filtering in the frequency domain is achieved by multiplying spectra together. Multiplication corresponds to adding logarithms and so a filtered spectrum can be derived by adding logarithmic (or dB) spectra. The procedure for cepstral analysis is to take the inverse fourier transform of the dB spectrum, thus converting the signal back into the time domain. This time domain signal is not a regular acoustic signal, since it was derived from the logarithmic spectrum; for this reason it is called a cepstrum, sort of a backward spectrum. The important property of the cepstrum is that since the dB spectrum is the sum of two spectra, so the cepstrum is the sum of two components corresponding to the source and the filter. Usefully the two components are separated: the lower end of the cepstrum corresponds to the filter while the higher end (or rather the middle of the reflected cepstrum) corresponds to the source. An illustration of the process of applying cepstral analysis is shown in Figure 7.4.

A further operation on the cepstrum is to remove the central part of the reflected cepstrum, the part that corresponds to the source, and perform a fourier transform to again generate a frequency domain version of the signal. Since the effect of the source has been removed from this spectrum, it shows the characteristics of the vocal tract filter. This is manifested as a much smoother spectrum than the original; the degree of smoothing depends upon the number of cepstral coefficients removed prior to the final fourier transform.

The technique of cepstral analysis is described in more detail in Harrington and Cassidy. The technique can be used to generate smoothed spectra which show the characteristics of the filter as described above. The cepstrum can also be used as a means of determining the fundamental frequency of voiced speech since the part of the cepstrum corresponding to the source is often manifested as a single spike. The location of this spike gives a measure of the frequency of the source signal.

Cepstral analysis is an important technique for examining speech signals. You should ensure that you understand the process of generating a smoothed spectrum and the reasons why the process works the way it does. Experimenting with the online examples may help you in coming to terms with it. This is just one of a family of techniques that seek to separate the source and filter in speech analysis. These techniques are useful in that they enable us to remove variability from the speech signal which is due to the source and concentrate our analysis on the vocal tract filter.

In the `Interactive Worksheets' on the Harrington and Cassidy CDROM, Chapter 3 presents some exercises on the source filter model. You should already have looked at the first three sections of this chapter in regard to filters. Section 3.4 covers the glottal source. Sections 3.5 and 3.6 cover tube models of the vocal tract and allow you to visualise the spectrum of a simple tube model as well as see the effect of combining source and filter spectra. You can also use this tool to hear the difference between the filtered and unfiltered source.

Cepstral processing is covered in the `Practical Exercises with Xlisp-Stat' worksheets, in sections 4.2 to 4.4. These worksheets demostrate how to generate a cepstrally smoothed spectrum in Xlisp-Stat and explore the effect of the number of cepstral coefficients retained in the analysis.

Chapter 3 of Harrington and Cassidy covers the source filter thoery in detail. Cepstral analysis is covered in Chapter 6.

The domain of this course is the related technologies of speech recognition and speaker verification. Speech recognition is the process of finding a linguistic interpretation of a spoken utterance; typically this means finding the sequence of words that were spoken but, as we'll see later, there is other linguistic information that we could recover, such as prosodic cues. Speaker verification is the problem of verifying that a `voice print' from an claimed user is the same as that stored for that user.

Both of these applications share some components. They must first pre-process the acoustic signal to parameterise it in a more useable and useful form. They must both match the input signal against a stored pattern and then make a decision about accepting or rejecting the match. Each problem has its own difficulties: in speech recognition the number of possible utterances can be very large and differences between speakers confound the problem; in speaker verification only 100% accuracy is really acceptable and differences in a speaker's voice due to the time of day or the microphone used can be as large as the differences between two similar speakers.

Most of this unit will look at speech recognition since this is both the most complex topic and the most used technology. Many of the techniques we learn about are applicable to the verification task as well.

The overall architecture of a speech recognition or speaker verification system is shown in Power's Figure 9.1. This diagram covers both speech and speaker recognition. Notice in particular the two paths through the system for training on the one hand and pattern matching or recognition on the other. The training process is vitally important and involves building some kind of internal model of the speech signal. In speaker verification there is one model for each speaker and the model must capture enough about the speaker's voice characteristics to be able to match that speaker and no-one else. In speech recognition the models might correspond to words, syllables, phonemes or other units. Much of what we'll study in this course is to do with what constitutes a model and how they are compared with incomming speech signals.

The general task of identifying who is talking can be described in terms of two different applications: speaker verification and speaker identification. In speaker verification (SV), an individual make a claim to his or her identity and we must verify that claim by matching a speech sample to a stored model. In speaker identification (SI), no claim is made and we must identify the speaker from a set of known speaker models. Speaker verification is used in security applications where a reliable method of identification is needed; for example for banking transactions over the telephone. Speaker identification might be used in a dialog system to allow tracking of multiple speakers in a dialog or to allow the system to recognise a user from earlier dialogs.

In order to verify a subject the input speech needs to be compared to a stored model for that user. The decision as to whether to accept the identity claim is based on a similarity threshold -- if the input is closer than the threshold, then we accept. However, this ignores all the other user's we know about so it is common to also take into account the similarity of the input to a group of `similar' users; we might say that if the input is closer to the claimed identity than it is to any of the similar users then we accept, otherwise we reject. As you can imagine, there are many variations on this theme.

One of the major considerations in SV applications in the reliability of the verification decision. The classic tradeoff in SV is between false acceptances and false rejections: if the threshold is relaxed, all of the valid claims will be accepted but so will many invalid ones; if the threshold is made more restrictive, most invalid claims will be rejected but so will many valid ones. In order to objectively evaluate a system's performance a measure called the Equal Error Rate (EER) is often quoted. EER measures the error rate of a system when the threshold is adjusted so that the number of false acceptances is equal to the number of false rejections. In a real application, the error performance can be adjusted to suit the level of security required: secure or convenient.

The speaker model used in SV or SI systems can be any kind of model of the acoustic signal. HMMs are common as is a method called vector quantisation; we'll discuss the details later. Unlike in speech recognition, the goal is to retain as much speaker specific information as possible in the signal.

No two utterances of the same word or sentence are likely to give rise to the same digitial signal; this obvious point underlies the difficulty in speech recognition but also means that we may be able to extract more than just a sequence of words from the signal. If we can understand the different sources of variability in the signal then we can begin to approach the problem of either separating them out for subsequent analysis stages.

Let's consider the factors which could cause two random speech samples to differ from one another:

The requirements of the signal modelling stage will differ depending on the application environment: telephone speech recognition applications may need to use techniques to compensate for line noise and changes in the properties of the telephone line during each utterance; an in-car system will need to be robust to significant amounts of background noise. These may call for different signal modelling techniques in addition to different approaches to pattern matching.

In order to understand LPC and many other signal processing techniques it is necessary to look at the mathematics behind the z-transform. Unfortunately this is quite a hairy topic for a non-mathematical audience and so can be a hurdle in properly understanding these techniques. In our book we try to give a simple step-by-step coverage of the z-transform and how it relates to digital filtering and I would like to walk through the major points in these notes. The aim of this exercise is so that you can look at, for example, equation 18 in Picone's paper and not only stay upright but maybe even say, "hmm, I see".

The story begins back in Chapter 6 on page 178. If you remember back to 801, we talked about frequency domain filtering as a mulitplication operation between the spectra of the source and filter. In this section we develop a new way of talking about the frequency domain, the z-transform, such that we can characterise a signal and a filter as a polynomial, and perform the filtering operation by multiplying these together.