The fastest way to digitize your handwritten notes!
We've all been there. It could be a note that you wrote down while listening to someone or a story that you had been writing since you were hardly 13. You got busy with your life like everyone else and forgot about that inspired story that you were writing. Suddenly one day it all comes rushing back to you and nostalgia hits you so hard that you want to finish that story as soon as possible. So you open that small wooden drawer that was shut since 2012 and you dust off the cobwebs and you crack a real smile after a very long time.
Now the challenge that you face is that you can't continue writing in a notebook since it's not a feasible solution anymore so you want it to be on your laptop and you want it digitized for free which makes sense since you wrote the story.
That is where I come in.
You can easily create a bot that shifts your handwritten notes to a digital document and the best part is that it wont cost you a dime!
Let me quickly explain about Uipath and CloudVision for the uninitiated.
Uipath: This is an RPA solution that is used to build automated solutions or "Bots".
CloudVision: This is a solution provided by Google and we will be using its API for analyzing the handwritten notes.
If you would like to test the Activity out, I recently published it on the Marketplace with additional features like Autocorrect and Bag of Words!
Step 1 - Prerequisites
Download and Install Uipath Studio and Register on Google for CloudVision API.
You can download Studio from here: https://www.uipath.com/
If you are completely new to UIPath, I will suggest you do a basic level of certification from Uipath Academy. It's free and easy.
You can register for Cloudvision here: https://cloud.google.com/vision/docs?authuser=2
Follow the "Get Started for free" registration and register. It's free for 1 year which should be enough for you to set up your bot and get your work done assuming you are working on a personal project. If it's an enterprise-level solution that you want to build, 1 year is enough for you to prove a POC and get funding!
Moving forward, once you have registered you will land up on this page,
Next, go to IAM and Admin and click on Service Accounts and click on Create Service Account.
Next, give a name to your service account. It will automatically create a service account ID. Click on Create. Your service account will be created!
Next, click on + Create Key, select JSON and click create. This will create a JSON file download it and save it on your workstation.
Also, save the Key ID in a text document.
These are important and will be used later on.
Step 2 - Building the BOT!
Now with the prerequisites taken care of let's build the bot!
Firstly, in the project folder, you need to download certain packages. Go to Manage Packages and add the following. They are:
Microsoft.Activities.Extensions
UiPath.GoogleVision.Activities
Bot Creation - Step 1 (Adding the File to be Read)
1. Add the activity:" Google Vision Scope". Set the following properties:
AuthenticationType : ServiceAccountKey
KeyPath : Give the Path to ServiceKey.json that you would have stored on the cloudvision page.
KeyType : json
2. Add the Activity "Handwriting Detection". Set the following properties:
Image Path : Path to the image.
ReturnedText : Create a string variable to store the Results.
Bot Creation - Step 2 (Cleaning the Data)
1. Create a new String variable strClearText and add the Activity "Assign". Add the following data:
strClearText = System.Text.RegularExpressions.Regex.Replace(strReturn,"[\n]"," ")This removes the new line character. You can add more filters here based on your requirements.
2. Write the cleaned data into a text file using the "Write Text File" activity.
Bot Creation - Step 3 (Doing a spellcheck)
No AI Model today is 100% accurate so to handle spelling errors we add a dialog popup that you can use to manually change words that the bot thought are misspelled.
1. Add the activity: "Invoke code". Add the following Arguments:
vDocPath : Path to text file
listErrorWords : list to hold the error words.After that add the following code:
Try
isreadonly = True
isVisible = False
missing = System.Reflection.Missing.Value
app = New Microsoft.Office.Interop.Word.Application()
doc = app.Documents.Open(fileName, missing, isReadOnly, missing, missing, missing, missing, missing, missing, missing, missing, isVisible, missing, missing, missing, missing)
Console.WriteLine("spelling errors-----------")
'check for typos
For Each err As Microsoft.Office.Interop.Word.Range In doc.SpellingErrors
Console.WriteLine(err.Text)
errorWords.Add(err.Text)
Next
'doc.CheckSpelling()
doc=Nothing
app=Nothing
System.GC.Collect
Catch ex As Exception
End Try This spell checks the data and returns a list of misspelled words.
2. Add the activity:"For Each" and loop through all the misspelled words. Add the following steps to get a dialog box to popup:
2.1 Add the activity "Input Dialog" and update as shown in the photo.
2.2 Add IF condition to skip words if they are as expected.
Finally, add the word to a dictionary using "Add to Dictionary" where the original word is the key and the corrected word is the value.
Bot Creation - Step 4 (Replacing the changed words)
1. Add the activity :"For Each" and loop through all the items of the dictionary.
1.Use assign activity to do a string search and replace.
strClearText = strClearText.Replace(item.key, item.value)2. Save the string in a text file.
Results!!!
First example:
This is a handwritten note with a decent and verbose handwriting.
Input:
Output:
Second Example: Taking a note that has a more natural flair to it:
Input:
Output:
As you would have seen, no results are a 100% accurate but it's pretty darn close and if done correctly won't take you more than an hour to setup nor will it cost you a single penny!! I think that's a win. Don't you?
Hope you enjoyed reading my article! If you have any questions feel free to reach out to me.
Share the love! :)