RPA tools and more ...

Once at work, I was given an R&D task to create a bot that would "walk" the site, select products, fill out forms and pay for purchases. At that time, we were writing a part of the Antifraud system that allowed detecting bots in the browser. And from that moment it all began ...





Table of contents

  1. RPA at a glance





  2. Open source projects





  3. Paid services





  4. Test Automation





  5. RPA vs Test Automation





  6. Website scraping and RPA





  7. BPM and RPA





  8. Safe RPA ...





  9. An example of a bot working in Python





  10. How to detect a bot?





  11. conclusions





RPA at a glance

RPA (Robotic process automation) - , ( , , .), , , ... - , . , .





:

RPA - , « » , , , . RPA -, .





RPA:





Destinations in RPA
RPA

Open source

open source , .. .

:





RPA open source
RPA open source

, , . Python , , .





Selenium & rpaframework

2 1 .. : , , . : , fingerprint . .





Selenium

Selenium WebDriver — -. Web-, . .

Selenium IDE - ; Chrome Firefox, .





RPA Framework

RPA Framework - RPA, Robot Framework Python. Selenium Playwright, Chromium, Firefox WebKit API. Robocorp .





3 in 1 (Desktop / Web / Mobile)

Robocorp

Robocorp RPA , . , .





TagUI

TagUI - RPA AI Singapore, . TagUI . , Windows, macOS Linux.





TagUI RPA
TagUI RPA

RPA Framework, Playwright, selector Selenium, .





Selenium RPA Framework

Selenium

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from webdriver_manager.chrome import ChromeDriverManager

driver = webdriver.Chrome(executable_path=ChromeDriverManager().install())
driver.get("https://www.google.com/")
elem = driver.find_element_by_xpath("/html/body/div[1]/div[3]/form/div[1]/div[1]/div[1]/div/div[2]/input")
elem.send_keys("Python news")
elem.send_keys(Keys.RETURN)
driver.close()
      
      



RPA Framework

from RPA.Browser.Playwright import Playwright
from Browser.utils.data_types import KeyAction

lib = Playwright()
lib.open_browser("https://www.google.com/")
lib.fill_text(selector="input", txt="Python news")
lib.keyboard_key(KeyAction.press, "Enter")
lib.close_browser()
      
      



RPA Framework API.





. Enterprise . , RPA . , , RPA 100 000 , .





RPA products
RPA products

RPA Everest Group RPA 2020:





Everest RPA , . Everest UiPath, Automation Anywhere, Blue Prism, Intellibot Nividous .





UiPath vs Automation Anywhere vs Blue Prism

Blue Prism, 2001 , RPA «Robotic Process Automation». UiPath UiPath «DeskOver». 2015 RPA-.



RPA , , :





VS
VS

Blue Prism . - , RPA.



, UiPath Automation Anywhere ( ) , . RPA «» , .



RPA - G2, . 21 :





  1. UiPath - 4.6 / 5 4722





  2. Automation Anywhere 4,5 / 5 4310





  3. Blue Prism 4,4 / 5 158





UiPath RPA?

UiPath RPA , . , RPA , , , AI Fabric, RPA SaaS .



UiPath RPA - 3-4 , RPA.





UiPath:





  1. Long Running Workflows





  2. Machine Learning and Predictive Analytics





  3. Seamless Interconnectivity





  4. Process Document Understanding





  5. Citizen Development





  6. Customer Satisfaction





  7. Flexible Licensing Model and Low Cost of Entry





: https://www.auxis.com/blog/top-rpa-tools





RPA G2, (, , .). G2 , :





G2 Grid
G2 Grid
RPA

UiPath

UiPath - .





UiPath . - . Quadrant Review Gartner, UiPath , UiPath , RPA.





GUI UiPath
GUI UiPath

Automation Anywhere

Automation Anywhere, RPA, - RPA, -. RPA, , , .





GUI Automation Anywhere
GUI Automation Anywhere

Blue Prism

Blue Prism, RPA, .





Blue Prism RPA Fortune. Blue Prism . , RPA. .





GUI Blue Prism
GUI Blue Prism

Microsoft Power Automate

Microsoft Power Automate RPA. Microsoft Power Automate . Microsoft . .





WinActor

WinActor - RPA, NTT Group. , .





GUI WinActor
GUI WinActor

Test Automation

.





, (, Selenium), . , , , , .





Automation Testing Tools

RPA.





:









Open source









Selenium





+









Appium





+









SoapUI





+









TestProject





+









Cerberus Testing





+









Katalon Studio









+





IBM Rational Functional Tester









+





Telerik Test Studio









+





TestComplete









+





Ranorex









+





Kobiton









+





Subject7









+





HPE Unified Functional Testing (UFT)









+





:





QA Automation tools
QA Automation tools

RPA vs Test Automation

: .





RPA Test Automation . , .





:





  • .





:





  • , , (SUT).





  • RPA , , , .





  • RPA , .





RPA

, , , RPA (, Octoparse).





Process Bots VS Search Bots

Process Bots VS Search Bots
Process Bots VS Search Bots

RPA:





  • Low Code UX





  • UX





  • -





  • -





  • - ( , ..)





:









  • -













  • HTML ( ; UPC )









  • (, , ..)





?





, , - RPA , , .





RPA . . . , , .





. . RPA . , -, RPA , , - html .





AI. , Crawlbot Diffbot, , . JSON CSV, . , , RPA. !





Scrape.do

Scrape.do - -, , API- - . Scrape.do . Scrape.do - -.





Scrapingdog

Scrapingdog - -, -, , CAPTCHA. HTML- - API. Scraping dog - API LinkedIn.





ParseHub

ParseHub - -, -. . , , , / , CSV JSON. .





Diffbot

Diffbot - -, , -. - . Analyze API , , , .





Octoparse

Octoparse - . IP- IP-. . , . CSV, Excel API.





ScrapingBee

ScrapingBee - . -, , Chrome.





Luminati

Luminati - - . , .





Scraper API

Scraper API - API -. -, CAPTCHA, HTML- -, API.





Scrapy

- Scrapy. Scrapy - , -. - Python.





Import.io

- -, , . Import.io - CSV. , 1000 API- . Mac OS X, Linus Windows.





BPM RPA

BPM (Business Process Management, -) – , -.





BPM - (, , ..).





BPM- — -.





, ,   . , , ,     , - .





- : ,   . RPA - ,  .





CAMUNDA
CAMUNDA
BPM RPA

Camunda

Camunda , , RPA- , .  , Camunda , RPA API / .





:









  • RPA





  •     RPA





  • RPA





ELMA

BPM- ELMA     . - ELMA  —  .           .





    RPA + BPM:





  •   .





  •   .





  •   .





  • Customer Experience     .





ProcessMaker

ProcessMaker - - (BPM) . .





RPA...

RPA





RPA? , . , ( , .), , , , . , .





, :





  • : . , .





  • : .





  • : , . , , , -.





  • : malware , .





  • : .





, , , .





:









  • : - , .





RPA





Let's look from the perspective of a pentester. We can use RPA to create a script that will cyclically check pages for vulnerabilities. Thinking further, we can train an entire army of robots to simulate attacks on various APIs and sites. It is clear that not everything can be automated, but you can try the basic scenarios that are repeated for each site.





To be continued...

In the next article, we will create our own bot, consider what algorithms for detecting bots are, which are written in Selenium and using RPA. Let's create our own algorithm and summarize.








All Articles