Contents for Recog
 Recog Help Contents

 Introduction

 Tutorials

 Menu Commands

 Demo Files

 Tricks and Tips


Contents - Introduction
 Recog Help Contents

 Introduction

What is Recog?

What are Neural Nets?

Is Recog like the Apple Newton?

Upgrades, bugs and legal stuff.

 Tutorials

 Menu Commands

 Demo Files

 Tricks and Tips


Contents - Tutorials
 Recog Help Contents

 Introduction

 Tutorials

How to do simple recognition.

How to make a new character.

How to let Recog learn on its own.

 Menu Commands

 Demo Files

 Tricks and Tips


Contents - Menu Commands
 Recog Help Contents

 Introduction

 Tutorials

 Menu Commands

File menu

Edit menu

Learning menu

Help menu

 Demo Files

 Tricks and Tips


Contents - Demo Files
 Recog Help Contents

 Introduction

 Tutorials

 Menu Commands

 Demo Files

ARROWS.RCG

NUMBERS.RCG

NUMBERS2.RCG

VOWELS.RCG

 Tricks and Tips


Contents - Tricks & Tips
 Recog Help Contents

 Introduction

 Tutorials

 Menu Commands

 Demo Files

 Tricks and Tips

Using Focus anticipation.

Undoing the auto-learn.

See the Neurons.


What is Recog?

Recog is a Windows program that learns and recognizes characters and symbols.  It uses neural networks to identify mouse-drawn characters.  You can easily teach it to recognize new characters, or it can even teach itself.  The program's accuracy improves as it learns your hand-writing (or mouse-drawing).

It is primarily an educational program.  Recog demonstrates the difference between supervised and unsupervised learning.  It has been used to introduce neural networks to students from grade one to the senior undergraduate level.

The best way to learn how to use this program is to follow one of the three short tutorials. The technical details relating to the neural networks behind Recog is not needed in order to use Recog.  The latest version of Recog can be freely downloaded from the Internet.


What are Neural Nets?

The underlying premise behind neural networks is that a large number of relatively dumb objects can exhibit intelligent behaviour when they are linked together in the right way.  Neural nets are composed of neurons which are linked to other neurons.  Each neuron is a simple device that totals up all the inputs it receives from other neurons, and if the total is above a fixed threshold, the neuron will fire and send inputs to all the neurons that are linked to it.  A neural network's flexibility comes from the configuration of the links, which can amplify, nullify or even negate the input signals.

Recog uses a large (but simple) two-layer neural network to learn and recognize patterns.  The hand-drawn image is digitized onto a grid of input neurons.  Each possible answer is represented by a single output neuron.  Every input neuron is linked directly to every output neuron (there are no hidden layers).  As in most neural networks, the data (or programming) is encoded in the links between neurons.  If a link between an input neuron and an output neuron is positive, that means that if the input is on then the total score for that output neuron is increased by a small amount.  If the link is negative, then it follows that if that input is on, the corresponding output has its score reduced by an amount.  The output neuron with the highest score (and thus the best match) is considered the winner.  This is known as a competitive network. To see the strengths or weights of the links between the input grid and a particular output, double-click on a character in the recognition list.

Recogs learning procedure is extremely simple.  There is no back-propagation, delta-rule, or other complexities; it is simple arithmetic.  All links between active input neurons and the selected output neuron have their weights increased by one.  All links between inactive inputs neurons and the selected output neuron have their weights decreased by one.  Double-click a character in the recognition list and watch how its weights change as Recog learns the character.


Is Recog like the Apple Newton?

Recog recognizes individually printed characters and gradually adapts to the style of writing that the owner uses.  This may sound suspiciously similar to Apple's ancient PDA (Personal Data Assistant), the Newton.

The Newton does not use neural networks, which makes it less flexible (and less volatile) than Recog.  This is advantageous since one doesn't want some random doodling to render the Newton illiterate.  The other key difference is that the Newton cheats by watching the order in which the strokes of the letters are made.  It watches as one writes, not afterwards.  This is a great idea for a PDA, but it obviously is no good for interpreting scanned text (such as that found in a book), and it certainly isn't the way humans recognize characters.

The PalmPilot is another PDA that recognizes handwriting.  But its recognition is hard-coded and unalterable.  There is no adaptation or learning on the part of the computer; it is the human that must adapt to the PalmPilot.


Upgrades, Bugs and Legal stuff

Upgrades:

This help file comes with Recog version 3.9 which was released in January 2001.  The most recent version of Recog (and its companion program BrainBox) can be downloaded from my web site.  Go to http://vv.carleton.ca/~neil.  If you have any comments or suggestions, I'd love to hear from you.  E-mail me at neil@vv.carleton.ca and mention that you have version 3.9.

Known Bugs:

None!

Legal stuff:

Recog is open source software.  You can copy, modify or delete it.  The VB3 source-code is included in the Recog package (RCG-SRC.ZIP) under the GNU Public License.  I spent a lot of time working on it, so my only request is that you e-mail me if you found something interesting to do with it.  Enjoy!


Tutorial 1: Simple Recognition

This tutorial will quickly demonstrate how to use Recog to recognize mouse-drawn characters.  Before starting, verify that the option None is selected in the Learning menu.

1)	The first step is to load the appropriate character network.  Go to the File menu and choose Open Character Net.  Open the file called VOWELS.RCG.  It contains the neural network needed to recognize the capital vowels A, E, I, O, U and Y.

2)	Next, use the mouse to draw one of the vowels.  Draw it in the large empty square underneath the label no picture file.  The left mouse button draws, the right mouse button erases, and double-clicking clears the picture.

3)	When you have finished drawing your vowel, click the Digitize button.  This will scan your picture and convert it to a grid of 48 pixels that the neural networks can interpret.  When it is done, click the Recognize button.

4)	Recog will immediately tell you the probability that the character you drew was an A, an E, and so on.  This is called a competitive algorithm: each possible vowel is comparing itself to what you drew, and the one the matches the best is the winner.  Remember that this network was trained on my handwriting, so even though it will probably pick the correct letter, the percentage may not be spectacularly high.

Try drawing, digitizing and recognizing several other vowels.  Try inputting an H and see what it thinks.  To train it to recognize new characters, try the next tutorial.


Tutorial 2: Supervised Learning

This tutorial will quickly demonstrate how to refine and add new characters to Recog's repetoir.  Before starting, choose the Supervised option in the Learning menu.

1)	Open the character net file NUMBERS.RCG.  This network will recognize mouse-drawn digits from 0 to 9.  Naturally it was trained using my handwriting.  I draw my 1s with large serifs on the top and bottom.  Try drawing a 1 as a simple vertical line, then digitize and recognize it.  It will probably rank the 1 at around 50% since a vertical bar doesn't match my version.

2)	To correct this, select the correct digit from the list (in this case the 1), and then click the Learn button.  When the Learn button is used, Recog modifies the neural net of the selected digit so that it more closely matches the pattern currently displayed on the grid of pixels.

3)	Repeat the process twice more (redraw the simple 1, digitize it and learn it).  Each time it should get better at recognizing that what you are drawing is a 1.  It is slowly forgetting about my handwriting and learning yours.

4)	Draw an X, digitize it and recognize it.  The low percentages should indicate that it hasn't any idea what that character is.  Click the Add button and enter X as the name of the new character.  This will create a blank character net; it hasn't done any learning yet.  Choose the X from the list and click Learn.  Feed it two more examples X (learning each one) and it should have a pretty good idea of how to recognize this new character.

This style of learning is called Supervised since you had to tell it every step of the way when to learn and when to add characters.  The next tutorial describes how to use Unsupervised learning.


Tutorial 3: Unsupervised Learning

This tutorial will quickly demonstrate how to let Recog learn on its own.  Before starting, choose the Unsupervised option in the Learning menu.

1)	Create a New Character Net from the File menu.  Select a width and height of 5 pixels.  This will favour a square aspect ratio.

2)	Draw an arrow pointing right, then digitize and recognize it.  In Unsupervised learning mode it automatically adds a new character.  Since it doesn't know the name of what you drew, it will assign the character a temporary label.  Draw arrows in the other three directions.  Each time it will add new characters.

3)	Keep drawing, digitizing and learning these four arrows.  It should start to consistently match all right arrows with the label ?01, and all left arrows with ?02, and so on.  Each time you click the Recognize button, it will determine if this is something entirely new (and create a new character) or if this matches something it has seen before (an learn the pattern onto that character).  Isn't this easier than supervised learning?

4)	The drawback is that it has now way of knowing what the real names for these symbols are.  Once it has seen a few examples you have to tell it.  Double click on ?01 on the list, and click the Rename button in the Info window that pops up.  Rename it to Right or East.  Do this for the other three arrows.

It is entirely possible that it may create two separate patterns for an arrow.  This is OK; just call one East1 and the other East2.  It is the boss, if it distinguishes between two isotopes of a character, it may do so.  We can ignore this difference and treat both as the same character.


File Menu
File Menu Commands

New Character Net...

Deletes all characters and starts a new recognition network.  You can choose the resolution of the new network at this time.

Open Character Net...

Opens an existing character recognition network.  These files have RCG extensions, and are text-based.  One can use a text editor like Notepad to hack the saved files.

Save Character Net

Save Character Net As...

Saves the recognition network to disk.

New Picture

Erases the hand-drawn picture.  Same as double clicking the picture with the mouse.

Open Picture...

Opens a previously saved picture.  Bitmap (BMP), Windows MetaFile (WMF) and Icon (ICO) file formats are supported and may be loaded.  Note that the digitization process only looks for black, other colours are treated as white.

Save Picture

Save Picture As...

Saves the current picture to disk.  Pictures may only be saved as bitmaps (BMP).  This command can be used (with Open Picture) to train more than one network using the exactly the same data (e.g. to determine which works better, a 6x8 grid or a 9x12 grid).

Exit

Quits Recog.


Edit Menu
Edit Menu Commands

Undo

Undoes the last action.  Some File Menu commands cannot be undone (especially Exit).  After an undo is performed, one can always Undo the Undo.

Copy character

Copies the highlighted character net and puts it on the Clipboard.  This is a mechanism for copying single characters between RCG files that are based on the same resolution.  The character net is in text format, and can be pasted into a text editor such as Notepad for hacking before being returned to Recog.

Paste character

Places the Clipboard's character net into the recognition list.  If a character of the same name is already in the list, you are asked if you wish to over-write it.

Copy picture

Copies the picture and puts it on the Clipboard.  The picture can then be pasted into other programs such as Paintbrush.

Paste picture

Places the contents of the Clipboard into the picture.

Options...

Opens the Options window.  Allows access to several important configuration settings.


Learning Menu
Learning Menu Commands

None

Deactivates all learning.  Only recognition can be performed.

Tutorial 1 describes how to do simple recognition.

Supervised

Manually add, delete and learn characters.

Tutorial 2 describes how to use supervised learning.

Unsupervised

Automatically adds new characters and learns to distinguish between them.  Allows adjustment of the threshold below which a new character will be created (Create limit), and the threshold above which learning will take place (Learn limit).

Tutorial 3 describes how to use unsupervised learning.


Help Menu
Help Menu Commands

Contents

Starts this help file on the initial Contents screen.

Search for help on...

Starts the help system's Search feature.

About Recog...

Displays program information.  See also: Upgrades, Bugs and Legal stuff.


Choosing a new Resolution

When a completely new network is created (from the File Menu) Recog allows one to specify the resolution of the new grid.  In order to choose the resolution one must consider two parameters: what amount of detail is desired, and what aspect ratio is desired.

Detail

If one is planning on recognizing a small number of quite distinct characters (such as A, E, I, O and U), the amount of detail need isn't great.  However if the characters are similar (such as a, e, i, o and u) one would need more detail to pick up the differences.  The more pixels in the grid, the finer the details that the nets can extract, but the more learning it will require to train them.

Aspect Ratio

Numbers and capital letters are about 1.5 times taller than they are wide.  Therefore to get the best results, a grid that is specifically designed for such letters should have a similar height to width ratio (4x6, 6x9 or 8x12 depending on the detail needed).  Other characters have one to one aspect ratios (such as arrows pointing in different directions, or polygons).


Demo File: ARROWS.RCG

ARROWS.RCG is a simple demonstration file that comes with Recog.  Its net contains eight arrows, each pointing in one of the 8 compass directions.  It is a demonstration that Recog isn't limited to recognizing alpha-numeric characters.

This file is very similar to the net that is constructed in tutorial 3.

The four demo files are: ARROWS.RCG, NUMBERS.RCG, NUMBERS2.RCG, VOWELS.RCG


Demo File: NUMBERS.RCG

NUMBERS.RCG was the original concept behind Recog.  It contains a net for each of the ten decimal digits.

This file is used and modified in tutorial 2.

The four demo files are: ARROWS.RCG, NUMBERS.RCG, NUMBERS2.RCG, VOWELS.RCG


Demo File: NUMBERS2.RCG

This is an enhanced version of NUMBERS.RCG.  It uses a higher resolution pixel grid and was trained using unsupervised learning.  This means that it has many isotopes of each digit (there are several entries for the same digit, each one specialized at recognizing a particular variation on the digit).

For more information on unsupervised learning, see tutorial 3.

The four demo files are: ARROWS.RCG, NUMBERS.RCG, NUMBERS2.RCG, VOWELS.RCG


Demo File: VOWELS.RCG

VOWELS.RCG is a simple network that recognizes the six uppercase vowels: A, E, I, O, U and Y.  It is interesting that Recog cannot recognize the lower-case vowels using the same 6x8 grid.  This is because most of them are circular, which means a higher resolution square grid (like 10x10) would work better.

This file is used in tutorial 1.

The four demo files are: ARROWS.RCG, NUMBERS.RCG, NUMBERS2.RCG, VOWELS.RCG


Using Focus anticipation

Recog was designed so that the focus automatically switches to the next item you are most likely to use.  For instance, when you finish drawing a picture, you probably want to digitize it.  When digitization is finished, you probably want to recognize.  Accordingly, the focus switches to the appropriate button in anticipation of your next action.  This means that you don't have to choose each button with the mouse.  When you finish drawing, hit ENTER twice.  Digitization and recognition will occur.


Focus

When a control, such as a button or text box, has the focus, this means that any typing on the keyboard will affect that control.  Only one control at a time can have the focus, just as only one window in Windows can be active (able to receive user input) at any given time.  The user can change the focus either by clicking on a different control or by using the TAB key to cycle among the controls of the active window.

Richard Mansfield, VB for Windows.


Undoing the auto-learn

Unsupervised learning will recognize a pattern, and if the highest candidate's percentage is above the learn-limit, it will automatically learn onto that character.  This means that the percentage it actually displays is artificially increased since that is the percentage after learning.  This makes sense in the unsupervised concept where its judgment is correct by definition.

However, sometimes you may be interested in what the percentage was before it learned.  To find out, simply choose Undo Auto-Learn from the Edit Menu.  The previous learning process will be undone, and the percentages will be returned to the original level.  After you have viewed the percentage, you probably want to Undo the undo or click Recognize so that it does the learn.


See the Neurons

Double-clicking on one of the entries in the character list will pop open a window which lists the connection weights between the input grid and the neuron representing that character.  If you want to explore the neural networks in greater detail, download the BrainBox program from my website.  BrainBox allows you to import a Recog network and view it in the conventional neurons and connections manner.


Options

The options window is available from the Edit Menu.  There are five important settings that may be configured from this window.  Changes to these settings are not retained when the program is exited.

Pen width:

This option controls the thickness of the line used when drawing a character.  The width is in pixels.  Note that erasing (right mouse button) uses a line twice as wide as drawing.  One may wish to reduce the pen width if one tends to draw characters that don't fill the entire image.  The picture shows two letters A.  The first is drawn with a pen width of 6; it will not digitize as well as the second one which is drawn with a pen width of 2.  If one doesn't fill the screen and digitizations tend to be bloated, try reducing the pen width.

Learn depth:

The Learn depth setting determines whether Recog is stubborn or naïve.  It determines the maximum weights that links in the neural networks can have.  Each time Recog learns a character, the weights are changed by 1 to match the current input.  If the maximum weights are large, it will take a lot of training to change Recog's behaviour.  If the maximum weights are small, Recog learns quickly but has a short memory meaning that a couple of (possibly incorrect) learnings can completely change its perception of a character.  The default value of 3 is on the naïve side.  Recog is meant to be an educational program so it is preferable to play with fast-moving volatile nets.

Show digitizing box:

The first step in the digitization process is to establish the boundaries of the image.  If this option is selected, a red box is drawn around the image indicating where the boundaries are and whether the aspect ratio limits affected them.  Since the digitization process only scans for black, the red lines are invisible and do not affect subsequent digitizations.