Speech:Run Train Master Script


 * Home
 * Semesters - Project Work by Semester
 * Information
 * Experiments - List of speech experiments

Project Notes

 * Unix Notes
 * Active Directory
 * Backups
 * Network Bridge
 * Speech S/W Installation
 * Speech Corpus Setup
 * Switchboard Data Notes
 * Experiment Setup
 * Scripts Page
 * Model Building
 * [ Step 1: Run a Train ]
 * Step 2: Create the Language Model
 * Step 3: Run the Decode

-

Added Spring 2014

This tutorial will show you how to use the new Master Script to successfully Run a Train, which ultimately will create the Acoustic Model

The process of running a train is now much easier. In the original tutorial  on how to Run a Train, there were 6 steps involved. In each of those 6 steps, there is a tedious amount of work needed and confusing instructions that waste a lot of time.

Now with a Master script to process all 6 of those steps in a MUCH more efficient and smoother process for the user, more experiments and faster results can be made.

=Detailed Guide for Running a Train= NOTE: IF THIS IS YOUR FIRST TIME, PLEASE READ THE ENTIRE POST BEFORE RUNNING SO YOU UNDERSTAND WHAT YOU'RE ACTUALLY DOING.

Part 1 - Running the Master Script to Configure Your Acoustic Model
By the end of running this script, you will have successfully built an Acoustic Model that can be trained on.

To start up the script, copy and paste this command into your Terminal (click for setup help):

 

You will first see some administrative information regarding the current version of the script and a high level description of what this script is doing.

Step 1
The second part of the script will first ask you what type of Experiment you want to run - MASTER or CHILD.

A Master Experiment is one that someone creates a new Experiment directory on the root Exp directory located on Caesar:  ... This type of Experiment is the one students have been making since the start of the Speech Recognition project.

'''BEFORE RUNNING ANY NEW MASTER EXPERIMENT (i.e: /mnt/main/Exp/0200), YOU MUST CREATE A NEW ENTRY ARTIFACT ON MEDIA WIKI. '''

When running this script and choosing to create a new Master Experiment, you'll be shown this message too - please don't just skip this step. Direct orders from Professor Jonas.

Click here  to see a guide on exactly how to do this.

A Child Experiment is one that someone creates inside an existing Experiment. This is new to as of the Spring 2014 class. This type of Experiment is used when you want to use the same tran files but want to change the Senone or Density values in the sphinx_train.cfg... This became common practice in Spring 2014 when the Modeling group found out that by changing these values, it can significantly impact the results of running a train and eventually a decode. The other main reason this feature was added was the large number Experiments the Spring 2014 class began running so quickly. Professor didn't want the Exp directory to grow at the rate it was, so instead we thought it would be better to just have the ability to create almost the exact copy of one Experiment, but before generating the pruning the dictionary, generating the phone and feats files you might want to adjust those configuration values, this is the best way to do it.

Your Input
If you type in 'm' and hit enter, you will be sent directory to Part 2. However if you type in 'c' to create a Child Experiment, then you'll be prompted to enter the Experiment number you'd like to put this into, then you'll be sent to Part 2.

Step 2
The second step will be to create the new Experiment directories. There isn't much user input here for either a Master or Child Experiment.

For a Master Experiment, we are automatically retrieving the next available Experiment Number from the /Exp directory and calling the  script which creates our directories and gives the logged in user read/write/executable permissions on those directories.

For a Child Experiment, we will be doing the same thing, however we'll call the  script as we are using a different.

Step 3
Part three is where we setup our sphinx_train.cfg configuration file. The Modeling group has done an amazing amount of work really getting to the nitty gritty of what this file actually does during the process of creating the phone and feats files we process later in the script. For both types of Experiment's we ask for the same input, however again like step 2, we call different scripts.

Here is a screen cap of this step asking for the two values you can specify:



As of March 2014, the only two values we are adjusting in this file are the Senone and Density values - I'm sure before the end of the semester we will be adjust more of the values in the configuration file.

But as you can see, the user was prompted for a Density value (has to be power of 2) and also a value for the Senone. NOTE: As you can see, both of these prompts are OPTIONAL - if you do NOT supply them, the Density will 8 and the Senone will be 1000.

For a Master Experiment, the script being called is the  - and for a Child,

Step 4
This step is where we are generating our transcripts from one of the corpus subsets we have located in. We also are doing a couple other very important tasks.

Before we type any input into the command prompt, we have to determine which corpus subset we're going to use.

Determine a corpus subset to use. As of March 27, 2014 we have:

/100hr /10hr /50hr /first_5hr /full /last_5hr /mini /mini2 /tiny

These directories represent different corpus subsets to use for various stages of making and testing models. Each of those directories contain both audio files and textual transcripts (though neither are in a format that we can use directly).

When you know which one you'll use, enter your input in the format shown on the prompt and below (only enter one choice):
 * first_5hr/train
 * mini/train
 * 100hr/train

Step 5
This step is complicated - not on your part, but we're doing a lot of gritty work inside the script. Our goals for this step are:
 * Create the Dictionary
 * Copy over the Filler Dictionary
 * Copy over the genPhones.csh script
 * Run the genPhones.csh script
 * Insert the phrase 'SIL' into the generated .phone file in alphabetical order
 * Run the make_feats.pl script

That's a mouth full! Luckily for you all of these processes are automated - you just have to sit back and watch the screen for output.

Part 3 - Run the Train
If you've made it here, you have successfully configured your Acoustic Model, the most important part of the entire Experiment.. You now need to run a Train on this model by running another perl script.

When the master_run_train.pl script ended, it showed you a path to navigate to. This path is your base Experiment directory where all of your directories are.

'''YOU MUST NAVIGATE TO THAT PATH SHOWN AT THE END OF MASTER_RUN_TRAIN.PL BEFORE DOING ANYTHING ELSE

Run the following in your Base experiment folder:

% /mnt/main/scripts/train/scripts_pl/RunAll.pl

The first thing the trainer does is verify that it has everything it needs to build a model. Checking to see if: Transcript list is valid, it can find audio files the transcript references and vice versa. The experiment dictionary contains all the words used in the transcript. All Phones used in the dictionary are defined in the .phone file.