Having spent 3 solid days on doing a crash-course VS 2005 training it's time for a little diversion. Finally I made the time for another post on programming the Tablet PC.
The tablet PC is very good in recognizing handwriting, it will even make sense out of the worst scribblings you can imagine, for instance my own handwriting.
The image shows the Tablet Input Panel (TIP) which is part of the tablet OS. It shows the textual representation of the ink, clicking a word shows a list of alternative results from the recognizer. In this post I will demonstrate how to do that in your own code.
I have a Winform with on it: a panel to scrible, a button to clear the panel, a button to recognize and a listbox to display the results of the recognition. Here's the code :
using
System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using Microsoft.Ink;
namespace
TabletPCdemo
{
public class InkPad : System.Windows.Forms.Form
{
private System.Windows.Forms.Panel panel1;
private System.Windows.Forms.ListBox listBox1;
/// Required designer variable.
private System.ComponentModel.Container components = null;
private System.Windows.Forms.Button buttonRecognize;
private System.Windows.Forms.Button buttonClear;
[STAThread]
static void Main()
{
Application.Run(new InkPad());
}
private InkOverlay ic;
public InkPad()
{
InitializeComponent();
ic = new InkOverlay(panel1);
ic.Enabled = true;
}
protected override void Dispose( bool disposing )
{
if( disposing )
{
if(components != null)
{
components.Dispose();
if (ic != null)
ic.Dispose();
}
}
base.Dispose( disposing );
}
private void buttonClear_Click(object sender, System.EventArgs e)
{
// Wait for pen input to complete
while (ic.CollectingInk);
ic.Ink.DeleteStrokes();
panel1.Invalidate();
}
private void buttonRecognize_Click(object sender, System.EventArgs e)
{
// Wait for pen input to complete
while (ic.CollectingInk);
// Any strokes drawn ?
if (ic.Ink.Strokes.Count == 0)
{
MessageBox.Show("No ink written");
return;
}
// Is this a tablet PC ?
Recognizers myLanguages = new Recognizers();
if (myLanguages.Count == 0)
{
MessageBox.Show("No recognizers isntalled !");
return;
}
// Get the recognizer for en-us
Recognizer myLanguage = myLanguages.GetDefaultRecognizer(0x0409);
RecognizerContext myContext = myLanguage.CreateRecognizerContext();
// Prefered results
WordList myWords = new WordList();
myWords.Add("Gekko");
myContext.WordList = myWords;
myContext.Strokes = ic.Ink.Strokes;
RecognitionStatus myStatus;
RecognitionResult myResult = myContext.Recognize(out myStatus);
if (myStatus == RecognitionStatus.NoError)
{
RecognitionAlternates myAlternates = myResult.GetAlternatesFromSelection();
listBox1.Items.Clear();
foreach (RecognitionAlternate ra in myAlternates)
listBox1.Items.Add(string.Format("Result : {0}, Confidence: {1}", ra.ToString(), ra.Confidence.ToString()));
}
myContext.Dispose();
}
}
It’s using the Microsoft.Ink namespace which is in the Microsoft.Ink.dll
In the constructor of the form a new InkOverlay object is created. This class has a large list of overloaded constructors. One of them takes a windows handle and another one accepts a windows control. It is important which constructor you are going to use as that will influence the level of code access security required to run the code. When you want to scrible on an ActiveX control there is only a windows handle available. When you want to scrible on a panel you should pass the control itself to the constructor. Your code will work just as well when you pass the panel’s handle but the rise in the cas will be enough to prevent your tablet code running in (the default security settings of) IE.
The InkOverlay object is not a managed object so you have to clean up when finished with it. The Dispose method of the form invokes the Dispose method of the inkoverlay. Now the inkoverlay is ready for use, having set it’s Enabled property you can scribble on the panel.
The InkOverlay object has an Ink property which holds the strokes drawn by the pen. Clearing the panel is a matter of deleting the strokes, calling DeleteStrokes will do just that. Checking the CollectingInk property in a loop the code first waits untill all ink input is processed. Usually this is instantanious but sometimes you will see that there is a lag between the movement of your pen and the ink actually drawn. The loop will make sure the ink is ready. Having deleted the strokes you have to invalidate the panel so it will be redrawn. As an empty panel.
When it comes to recognizing handwriting most tablet PC code examples show the oneliner:
ic.Ink.Strokes.ToString();
The Strokes class has overloaded the ToString method to return the most probable recognition result. The code given here will return all results, like the TIP.
Before recognizing it performs two checks, to prevent exceptions. First of all there have to be some strokes to recognize. Makes sense, but you’ll get an exception trying to recognize an empty strokes collection. Next the code checks if there are any recognizers installed. You can install the ink assemblies on a non Tablet PC. Your app will work fine, using the mouse you can even scribble. But the recognizers are only installed on a real tablet. There are several recognizers available, for English, German, French, Chinese, a couple more and a couple more announced. The code has to pick the recognizer for a specific language using the GetDefaultRecognizer method. The parameterless overload checks for the language of the tablet the app is running on. But it does this in a (imho) buggy manner. Experimenting shows it checks for the regional settings and not the language settings. I’m running the English version of the Windows Tablet PC with my regional settings set to Dutch. The result is that GetDefaultRecognizer starts looking for a Dutch recognizer. Which is, alas, not (yet) available. To select the recognizer to use I pass in an LCID (locale ID) constant. This is a point where the COM base of the tablet API really shines through.
To use the recognizer you need its context, which is created by the CreateRecognizerContext method. I want to steer the recognizer’s result and do so by adding a list of favored words to the context. Create a new WordList object and add your favorites; for the sake of demo I’ve only added “gekko” (the Dutch spelling of gecko) to mine. The strokes to be recognized are added to the context and a RecognitionResult object is created. Now all is set up and the Recognize method will perform the actual recognition, returning the result in the recognition object.
Now I can read all recognition results, in order of probability, from the result using the GetAlternatesFromSelection method. Besides the text recognized also the Confidence the recognizer has can be read from the RecognitionAlternates. Don’t forget to dispose the context, it’s also an unmanaged resource.
You can see that even if the recognizer got my bad writing right it does not have that much confidence. But my Gekko is favoured.
Far better when I really do my best.
