How to share reusable code and functionality between your robot projects using Git submodules?

NOTE: This blog post contains outdated information. Please follow instructions in the documentation for sharing robot code.

A separate Git repository for the shared code

One option for code-sharing is to store the shared code in a separate Git repository and then include the code into the robots that need it using Git submodules.

"It often happens that while working on one project, you need to use another project from within it. Perhaps it's a library that a third party developed or that you're developing separately and using in multiple parent projects." - Git documentation

Let's consider this shared robot code repository:

https://github.com/robocorp/example-shared-robot-framework-keywords-and-libraries

This example robot code repository contains shared code that other robot projects can import and use.

The shared code repository looks like a normal robot project. Why is that?

The shared robot code repository looks like a normal robot project repository because it is a normal robot project. :)

The shared code project was created using RCC and the extended Robot Framework template. The project is self-contained and can be run as a normal robot project. The shared keywords and libraries can be developed and tested in isolation.

When developing shared code, you want to be able to test it. Having the shared code as a normal robot project makes it easy to execute the keywords, call the library methods, etc., without unnecessary external dependencies. The project documents its dependency requirements in the conda.yaml file.

An alternative approach would be to set up another project for testing the shared code. That test project would include the shared code repository as a Git submodule. Then you would have something like this:

A test repository that includes the shared code repository as a Git submodule.

A real production robot repository that includes the shared code repository as a Git submodule.

Another real robot repository using the shared code.

One more production robot repository using the shared code.

Whatever the repository configuration, make sure to test the shared code in isolation to make sure it works!

An example of including the shared code using Git submodules

The shared code repository is used as a Git submodule in another example repository: https://github.com/robocorp/example-use-git-submodule-for-shared-code.

Here's what the file structure looks like in the project that includes the shared code as a Git submodule:

shared @ 17dff6b
.gitignore
.gitmodules
README.md
conda.yaml
robot.yaml
tasks.robot

The shared directory in the above file structure is the shared robot code repository included as a Git submodule.

The https://github.com/robocorp/example-use-git-submodule-for-shared-code repository that includes the shared code repository was created using RCC and the standard Robot Framework template.

The shared Git submodule in this project was created with the git submodule add command:

git submodule add https://github.com/robocorp/example-shared-robot-framework-keywords-and-libraries shared

The .gitmodules file defines the path and the URL to the included repository:

[submodule "shared"]
    path = shared
    url = https://github.com/robocorp/example-shared-robot-framework-keywords-and-libraries

See the tasks.robot file in the repository for examples of importing and using the shared code.

How to clone this example project, including the Git submodule?

To clone this project, including the Git submodule, use the following command:

git clone --recurse-submodules git@github.com:robocorp/example-use-git-submodule-for-shared-code.git


The --recurse-submodules option handles cloning the submodule. See the Git submodule documentation for more information.

How to pull the upstream changes to the submodule?

To update (pull the upstream changes to) the submodule contents, use the following command:

The Git submodule strategy works for code-sharing, but there are some caveats. Robocorp Cloud does not automatically clone the submodules at the time of writing. This means you can not use the Git repository linking feature in Robocorp Cloud for robots that use Git submodules.

However, when using Git submodules, you can still upload your robot to Robocorp Cloud using the upload functionality in Robocorp Lab, VS Code, or RCC. Since the upload creates a zip package with the project contents, the zip also contains the submodule contents!

Remember to keep the shared code version in sync if you have multiple developers working on your robot projects. Since the submodule does not update automatically when you pull changes to your main robot project you need to remember to run the git submodule update --remote command manually. Creating a shell script (containing the required pull commands and options) that each developer can run to update the robot code repository might be a good idea.

Read the Sharing libraries article to learn about other code-sharing possibilities!

Robocorp work data management - build robust, scalable, high-performance automations on top of Python-based open-source RPA

"Her hands are sweaty, morale weak, mouse feels heavy

There's tomato sauce on her sweater, granny's spaghetti

She's tired, but on her lap, she has her teddy"

Once upon a time, there was an office worker. Her job was great and fulfilling most of the time. Except at the end of each fiscal year. This is her story.

"It's as easy as catching salmon with your bare hands. Bears do it all the time!"

"Let me explain the process, so you know how to complete it, ok?" her boss interrupted.

"Each year, we collect some very important data from various systems and then enter it into our company ERP." her boss explained.

"And that's it. Good luck!" her boss smiled and hurried out of the door.

Luckily, she managed to find someone in the office who explained the process in a bit more detail.

"Ok. First, you navigate to Nasdaq's website and download this Excel spreadsheet." the helpful person explained.

"There, you will find a few company stock tickers."

"Yes! Five thousand tickers!" the helpful person smiled happily.

After a few moments of contemplating her life choices, she gathered herself and asked:

"And what do I do with that handful of tickers?"

"Glad you asked!" the helpful person exclaimed.

"You navigate to Yahoo Finance's website. For each ticker, you open the page for that ticker and take a screenshot of this market data." she demonstrated.

"For five thousand tickers?" she verified with a hint of fear in her voice.

"For five thousand tickers!" exclaimed the always so cheerful person.

"Now you have half of the required data." the helpful person explained.

"Where's the other half?" she queried worriedly.

"No worries, my friend! The rest of the data can be found on another site: https://www.annualreports.com/."

"Dare I ask in what format that data is on that site?" she asked, fearing the answer.

"You are in luck! The data is beautifully laid out in PDF documents!"

She felt her heart skip a beat. No. Not PDF documents. Anything else but those.

"Inside those documents, you look for this specific heading and then copy and paste the data from that page." the helpful person continued.

"Is the data always on the same page in each of the documents?" she asked, not sure if she was ready to handle the response.

"No, of course not!" the other person laughed.

"Sometimes it's on the first page, sometimes on any other page!"

"Ok. And how many pages are in one document?"

Cold sweat drenched her back. After picking herself up from the floor, she managed to continue:

"And now that I have the market data screenshots and the company detail PDF documents, then what?"

"Then, you log in to our company ERP application and create a sales order for each company."

"You export each sales order as a PDF file."

"Finally, for each company, you email the market data and the company details and attach the PDF files."

"And how long does this process usually take...?"

"Oh, it needs to be completed by the end of the year, so if you start in September, you should be fine!"

"September? That's... four months?" she calculated, fighting away the tears of panic.

"Yes, give or take. Good luck!" the helpful person exclaimed happily and disappeared to her cubicle.

Back at her desk, wiping her sweaty palms, tiredly reaching for her mouse that felt exceptionally heavy that morning, glancing at the delicious spaghetti sauce stain on her sweater, she placed her teddy bear on her lap for mental support and navigated to Nasdaq's website.

Fetch thousands of stock tickers from the Nasdaq website as an Excel spreadsheet. (This can be done using browser automation.)

For each ticker, get a screenshot of the market data from Yahoo Finance's website. (This can be done using browser automation.)

For each ticker, download the company info PDF document from https://www.annualreports.com/. (This can be done using browser automation.)

For each company, find the relevant data from the market data screenshots. (This can be done programmatically using OCR (optical character recognition). For OCR, one can leverage intelligent automation and machine learning by integrating with cloud service providers, use de facto open-source OCR solutions such as Tesseract, or implement a custom solution to target the exact needs of any use case).

For each company, find the relevant page and data from the PDF documents. (This can be done by writing some PDF data extraction code in Python.)

Input the required data from the screenshots and the PDF documents to the Windows-based ERP application. (This can be done using Robocorp Workforce Agent and Windows automation.)

Export each entry from the ERP as a PDF document. (This can be done using Windows automation.)

Email the PDF documents to the recipients. (This can be done using email automation.)

That's a lot of work for any person to handle! Not even a teddy bear is probably enough.

Maybe we can help the poor office worker in distress? Let's see how the same process could be automated with Robocorp.

We could split the process into three distinct tasks.

The first task fetches the tickers from Nasdaq.

The second task collects all the required data for each ticker.

The third and final task enters the data into the ERP system and sends out the verification emails.

The first and the second task can be run in a Control Room container. The advantage of that is you do not need to host the run environments yourself. It just works! Let your IT department take the afternoon off and enjoy some carbonara instead.

The third task requires interacting with a Windows-based desktop application. It can be run in a Windows environment using Robocorp Workforce Agent. Robocorp also supports macOS and Linux!

Enabling high performance and error handling with work items and parallel execution

The second step, collecting all the required data for each ticker, involves taking thousands of screenshots, downloading thousands of humongous PDF documents, and finding and scraping data out of the images and documents.

Even when the robot goes as fast as it can, it is restricted by the speed of the systems where it gathers the data from. Sure, it can work more quickly than a human (or match the speed of a teddy bear - those things are fast!) but is there a way to improve the speed even further?

Luckily, there's a built-in solution in Control Room to address that challenge.

To speed up the task execution, we can configure Control Room to use parallel processing. Instead of collecting the company data one ticker at a time, we can simultaneously collect data for many companies. It's like an army of teddy bears!

To enable parallel processing, we should split our work into small chunks, work items, so that each of those items can be processed separately. In this case, one ticker becomes one work item.

We can tell our robot to process multiple work items at the same time using the parallel execution support in Control Room. This results in faster process execution. Neat!

In addition to enabling parallel processing, work items have other advantages, too.

We can monitor the status of work flowing through the process in real-time. Piece by piece, work gets completed. Captivating!

If something goes wrong when processing a single work item (for example, when some data is invalid), that item can be marked as failed. The processing can then continue with the rest of the work items - no need to halt the whole caravan due to one camel running out of humps.

The failed work items can be manually handled and re-run later without re-running everything. This way, you can avoid processing already successfully processed items again.

So, instead of running all the data gathering for the thousands of tickers again, we can fix the data in the failed work items and re-run only those items.

This task involves entering the data into the Windows-based ERP system and sending out the verification emails.

Conveniently, as already mentioned, Robocorp supports multiple execution environments. In addition to Control Room containers, you can run tasks in other environments, too.

In this case, the robot needs to interact with a desktop application. This scenario is supported by installing the Robocorp Workforce Agent on the Windows machine.

The Robocorp Workforce Agent handles downloading, installing, and running the robot on the Windows machine.

This way, the robot gets access to the ERP application and can start entering the data, exporting and saving the entered data as PDF documents, and sending out the verification emails with the PDF documents attached.

The advantages of Robocorp, work data management, and work items

Work data management and granny's spaghetti

How to automate a desktop application with Robocorp? What automation strategies are available? What are the best practices? Image locators, keyboard navigation, OCR, accessibility locators.

Desktop automation imitates a human operator controlling a desktop interface. This includes opening and closing applications, simulating mouse movements and clicks, plus triggering keyboard keys and shortcuts.

Let's look at the following options for desktop automation:

Image locators: Using screenshots to find elements

Keyboard navigation: Using key combinations, typing text

OCR (optical character recognition): Reading text from the screen

Textual locators: Accessing the elements by their accessibility names and other attributes

The image locator strategy is based on taking screenshots of the graphical user interface (GUI) elements. The robot uses those captured screenshots to find the elements on the screen. The robot is instructed what to do when an element is found or not found. The action could be, for example, clicking on the element when it appears or waiting for it to disappear before continuing.

The following animation displays a robot using VS Code (with Robocorp extensions) user interface, image locators, keyboard, and OCR to create and run a new robot(!):

This is what the robot task looks like in Robot Framework syntax:

*** Tasks ***
Create and run a robot using VS Code, image locators, keyboard, and OCR
    ${new_robot_dir}=    Create new robot dir
    Open VS Code    ${new_robot_dir}
    Open Command Palette
    Create robot
    Select standard template
    Enter robot name
    Open Command Palette
    Run robot


This robot really exists! You can view the robot code at https://github.com/robocorp/example-vs-code-robot.

Image locators require a screenshot of an element or any section of the screen. For example, this Run a Command... widget on VS Code welcome screen:

It is possible to give a name for the image locator. The locator metadata is stored in JSON format. Using the Run a Command... screenshot as an example, the JSON for a locator called VSCode.RunCommand looks like this:

The robot code using the VSCode.RunCommand locator looks like this:

*** Keywords ***
Open Command Palette
    Wait and click    alias:VSCode.RunCommand


Image locators can target any visual elements on all major platforms (Windows, Linux, macOS).

Image locators work without physical access to the application or the operating system (e.g., Citrix) as long as an image of the screen is available. The operating system or the application under automation does not matter.

Image locators help determine the completion of actions based on what is or is not on the screen: a button becomes visible, a progress bar disappears, etc.

Image locators fail if the visuals change too much (layout, fonts, icons, colors, resolution).

Multiple screenshots of the same elements are needed if the locator needs to work with elements that have multiple states (e.g., an active button vs. an inactive button) since a different color or background color often conveys the element state.

The application state (and thus the GUI) might differ on application start requiring some robot logic for resetting a known start state.

Opening windows and dialogs, executing tasks, and navigating can be done without a mouse if the application provides access to all the core functionality using the keyboard.

This robot keyword uses a shortcut key combination (Ctrl+Shift+P) to open the VS Code Command Palette:

*** Keywords ***
Open Command Palette
    Press Keys    ctrl    shift    p


This keyword runs the robot by typing the run command and pressing Enter:

*** Keywords ***
Run robot
    Type and run    robocorp-run-robot


Keyboard shortcuts change less often than visual user interfaces, making them more robust than image locators. Even if the screen resolution, colors, or fonts change, keyboard shortcuts usually stay the same.

Keyboard shortcuts are easy to test manually, making development faster compared to creating image-based locators.

All applications do not have sufficient keyboard support.

Keyboard shortcuts are excellent for triggering actions, but there are no shortcuts for determining when a slow action has been completed.

Keyboard navigation needs to be used together with either image or OCR locators to assert the completion of slow operations.

OCR locators work by extracting and matching text on the screen. The locator can be written in plain text (e.g., "Submit"). The robot will find the text on the screen programmatically.

This keyword uses OCR (to read text from the screen) to determine when a slow action has been completed. After executing the robot creation command on the VS Code UI, the robot waits for the environment update notification (Update env) to appear and then disappear to indicate the completion of the action:

*** Keywords ***
Create robot
    Type and run    robocorp-create-robot
    Wait for text    Update env
    Wait for text to disappear    Update env


An OCR locator is plain text and thus easy to implement.

OCR locators can be somewhat more resilient against some visual UI changes than image locators (resolution, layout, color).

OCR locators can fail due to a lack of contrast or UI effects such as shadows and transparency.

OCR locators require unique matches. If there are no unique texts on the screen, one needs to target specific parts of the screen to narrow down the results, making the locator less resilient against visual changes.

Textual locators (Windows accessibility locators)

Some Windows applications can be automated by targeting the actual UI elements within them, referring to them by their textual UI accessibility identifiers.

Accessibility locators are resilient to visual changes. The identifiers usually stay the same across operating system versions.

Not all Windows applications can be inspected for accessibility locators.

If the application is accessed remotely using Citrix or a similar protocol, the accessibility identifiers are unavailable.

Instead of waiting an exact amount of time between actions, utilize the intelligent waiting capabilities. Most of the locators support some sort of "wait for this element" functionality. Using the built-in waiting makes the robot wait for the elements just long enough before proceeding.

Prefer strategies that do not rely on the exact visual representation of the graphical user interface. Using keyboard shortcuts and accessibility locators might produce more robust automation compared to image locators and OCR.

If possible, try to run the desktop automation in an environment where you can make sure the application configuration and visuals remain unchanged.

In practice, a combination of multiple strategies is needed. Use the strengths, be mindful of the weaknesses, and combine the strategies to build a robot that does what it's supposed to do and is quite resilient against moderate changes in the UI.

Robocorp is an open platform based on Python. This makes it possible to import and use the libraries you want or even implement your own. Out-of-the-box support is provided for all the automation strategies, but you are not limited to those. Use the power of the Python ecosystem!

Desktop automation strategies and best practices using image locators, keyboard navigation, OCR, and accessibility locators

You may have seen that Robocorp recently completed our SOC 2 Type II certification. There’s a lot of buzz around the SOC 2 certification, not just from Robocorp, but from companies across the board. So I wanted to dig a little deeper and explore just what the SOC 2 certification means.

SOC - Scottish Ornithologists' Club? Not quite. System and Organization Controls.  

According to Vanta, “A SOC 2 audit is an independent, third-party assessment of your security practices, and it can be a great way to grow your business and assure larger customers of your security.” 

As you may know, the SOC 2 audit comes in two flavors:

Type I: Collect data for one day. Show you understand the necessary security procedures.

Type II: Prove you follow those procedures over a long period of time.

Everything worth doing is worth doing well. Ever since its inception, Robocorp has followed operational best practices and is committed to continuing doing so, including privacy and data security.

Ok, Robocorp. You talk the talk, but do you walk the walk?

Although we speak with integrity and honesty, getting additional assurance of dedication to quality from external sources does not hurt. That is why we have worked with an external auditor with the goal of hanging a shiny “SOC 2 Type II Certification” on our virtual office wall.

Also, it is really hard selling to enterprises if you haven’t paid for pieces of paper. ;-)

The certification proves that we understand the necessary security procedures and seemingly do so consistently!

How does SOC 2 apply to Robocorp in practice?

Sure, there is quite a bit of work required on the road to compliance, but in Robocorp’s case, not many practical changes were needed due to our continuous dedication to operational best practices.

Robocorp partners, customers, and users can rest assured that their data will be handled responsibly, professionally, and with great care. Whether it’s orchestrating and managing the robots and their business data in Control Room or using the developer tools, security is always taken into consideration.

Most of the effort went to documenting and formalizing policies and training Robonauts.

Robocorp uses Vanta for active, real-time monitoring and alerts, also after receiving the certification. Thanks to this, the compliance stays great constantly and does not rely solely on annual checkpoints.

How long does it take to get SOC 2 Type II certified?

It depends. In Robocorp’s case, the practices and policies were already on a good level. However, we wanted to ensure that these practices are here to stay, so all in all, the process took roughly six months to go through the thorough audit performed by a third party.

To learn more about Robocorp’s SOC 2 certification, I encourage you to read our announcement. Or to learn more about our commitment to security, visit robocorp.com/security

SOC it 2 me: Jani explains SOC 2

Learn skills for life. Robocorp is built on best practices, on widely popular and relevant technologies, projects, and concepts.

"Of course, Isabella. What is on your mind, my dear?"

"How did you know you wanted to become a veterinarian?"

Grandma followed a lonely seagull in the distance with her gaze before answering.

Isabella dug her fingers into the warm sand, sitting in silence, staring at the waves gently hitting the shore.

"Is something bothering you, my child?" grandma asked softly, knowing the answer already.

"This is the last year of school. Dad wants me to go to the university and study law."

The seagull swooped down towards the surface of the ocean.

Grandma, tracking the diving bird with her eyes, smiled warmly.

Isabella lifted her gaze and looked at grandma almost apologetically.

"Lawyers do important work. They are respected. They get paid well." Isabella muttered unenthusiastically.

With a splash, the seagull rose towards the sky, a plump fish in its beak.

"Those are all true." grandma agreed tenderly.

A screech of disappointment echoed over the waves as the lucky captive escaped its captor.

"I've lived a long life, Isabella. I went through many a profession. We were young, grandpa and me. We did what we had to to get by. To pay the bills. To pay for the house. But there was no passion in those endeavors."

"Taking care of animals was and is my passion. The studies took time, of course, meaning I spent the days working and the nights studying. It was hard. And it was the best decision of my life. To finally dedicate my life to something I truly loved."

Isabella sat in silence. Pondering the words.

Blog

Robocorp offers its clients the ability to build, scale, deploy, and maintain their robot workforce.

All posts by Jani Palsamäki

How to share reusable code and functionality between your robot projects using Git submodules?

Work data management and granny's spaghetti

Desktop automation strategies and best practices using image locators, keyboard navigation, OCR, and accessibility locators

SOC it 2 me: Jani explains SOC 2

The World on the Other Side

Combining the power of open-source projects and communities

Why pay many money when few money do trick?

Automation vendor lock-in: How to deal with it?

products

solutions

developers

digital workers

resources

company