Extract Text With OCR in Power Automate Desktop

As the Robotics Process Automation market matures, organizations have expectations that their automated solutions become more intelligent. Automated solutions become more intelligent using Artificial Intelligence (AI). One discipline of AI is the use of Optical Character Recognition (OCR). This capability allows you to ingest a binary object (like an image or document) and then extract the related text from that image.

Microsoft provides OCR capabilities inside of AI Builder. This is a good option for people who do not have access to Power Automate Desktop but need those capabilities. In addition, Microsoft has recently included a new action in Power Automate Desktop called Extract Text with OCR that allows us to extract text from a web page or image.

Let’s take a closer look at this feature:

1. We need to create a new Desktop flow and then add the Create Tesseract OCR engine action onto the Desktop flow. We will select English as our language and then leave the Image multipliers.

Note: A variable will be produced called OCREngine that we will reference in our next action.

2. We will now drag the Extract text with OCR action onto our Desktop flow. We will set our OCR engine variable to %OCREngine% that was created in our previous step. We will also change our OCR source to be Image on disk and provide a static path to our Image file path. When the action runs, it will create a variable that we can rename to be BroncoSport.
   

Testing

I downloaded an image from the https://www.ford.com website of a vehicle that contains some text. Let’s see if we can extract the text from this image.

When we run our Desktop flow in debug mode, we will find the following:

Conclusion

When it comes to Artificial Intelligence, it continues to evolve. Having expectations for 100% accuracy isn’t realistic at this point. However, as we saw in the previous example, it is still quite impressive. But, it is really important to have fall back strategies and provide assertions against AI outputs to ensure of desired results.