Once at work, I was given an R&D task to create a bot that would "walk" the site, select products, fill out forms and pay for purchases. At that time, we were writing a part of the Antifraud system that allowed detecting bots in the browser. And from that moment it all began ...
Table of contents
RPA at a glance
Open source projects
Paid services
Test Automation
RPA vs Test Automation
Website scraping and RPA
BPM and RPA
Safe RPA ...
An example of a bot working in Python
How to detect a bot?
conclusions
RPA at a glance
RPA (Robotic process automation) - , ( , , .), , , ... - , . , .
:
RPA - , « » , , , . RPA -, .
RPA:
Open source
open source , .. .
:
, , . Python , , .
Selenium & rpaframework
2 1 .. : , , . : , fingerprint . .
Selenium
Selenium WebDriver — -. Web-, . .
Selenium IDE - ; Chrome Firefox, .
RPA Framework
RPA Framework - RPA, Robot Framework Python. Selenium Playwright, Chromium, Firefox WebKit API. Robocorp .
3 in 1 (Desktop / Web / Mobile)
RPA Framework, Playwright, selector Selenium, .
Selenium RPA Framework
Selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install())
driver.get("https://www.google.com/")
elem = driver.find_element_by_xpath("/html/body/div[1]/div[3]/form/div[1]/div[1]/div[1]/div/div[2]/input")
elem.send_keys("Python news")
elem.send_keys(Keys.RETURN)
driver.close()
RPA Framework
from RPA.Browser.Playwright import Playwright
from Browser.utils.data_types import KeyAction
lib = Playwright()
lib.open_browser("https://www.google.com/")
lib.fill_text(selector="input", txt="Python news")
lib.keyboard_key(KeyAction.press, "Enter")
lib.close_browser()
RPA Framework API.
. Enterprise . , RPA . , , RPA 100 000 , .
RPA Everest Group RPA 2020:
Everest RPA , . Everest UiPath, Automation Anywhere, Blue Prism, Intellibot Nividous .
UiPath vs Automation Anywhere vs Blue Prism
Blue Prism, 2001 , RPA «Robotic Process Automation». UiPath UiPath «DeskOver». 2015 RPA-.
RPA , , :
Blue Prism . - , RPA.
, UiPath Automation Anywhere ( ) , . RPA «» , .
RPA - G2, . 21 :
UiPath - 4.6 / 5 4722
Automation Anywhere 4,5 / 5 4310
Blue Prism 4,4 / 5 158
UiPath RPA?
UiPath RPA , . , RPA , , , AI Fabric, RPA SaaS .
UiPath RPA - 3-4 , RPA.
UiPath:
Long Running Workflows
Machine Learning and Predictive Analytics
Seamless Interconnectivity
Process Document Understanding
Citizen Development
Customer Satisfaction
Flexible Licensing Model and Low Cost of Entry
: https://www.auxis.com/blog/top-rpa-tools
RPA G2, (, , .). G2 , :
RPA
UiPath
UiPath - .
UiPath . - . Quadrant Review Gartner, UiPath , UiPath , RPA.
Automation Anywhere
Automation Anywhere, RPA, - RPA, -. RPA, , , .
Blue Prism
Blue Prism, RPA, .
Blue Prism RPA Fortune. Blue Prism . , RPA. .
Microsoft Power Automate
Microsoft Power Automate RPA. Microsoft Power Automate . Microsoft . .
WinActor
WinActor - RPA, NTT Group. , .
Test Automation
.
, (, Selenium), . , , , , .
Automation Testing Tools
RPA.
:
|
Open source |
|
Selenium |
+ |
|
Appium |
+ |
|
SoapUI |
+ |
|
TestProject |
+ |
|
Cerberus Testing |
+ |
|
Katalon Studio |
|
+ |
IBM Rational Functional Tester |
|
+ |
Telerik Test Studio |
|
+ |
TestComplete |
|
+ |
Ranorex |
|
+ |
Kobiton |
|
+ |
Subject7 |
|
+ |
HPE Unified Functional Testing (UFT) |
|
+ |
:
RPA vs Test Automation
: .
RPA Test Automation . , .
:
.
:
, , (SUT).
RPA , , , .
RPA , .
RPA
, , , RPA (, Octoparse).
Process Bots VS Search Bots
RPA:
Low Code UX
UX
-
-
- ( , ..)
:
-
HTML ( ; UPC )
(, , ..)
?
, , - RPA , , .
RPA . . . , , .
. . RPA . , -, RPA , , - html .
AI. , Crawlbot Diffbot, , . JSON CSV, . , , RPA. !
Scrape.do
Scrape.do - -, , API- - . Scrape.do . Scrape.do - -.
Scrapingdog
Scrapingdog - -, -, , CAPTCHA. HTML- - API. Scraping dog - API LinkedIn.
ParseHub
ParseHub - -, -. . , , , / , CSV JSON. .
Diffbot
Diffbot - -, , -. - . Analyze API , , , .
Octoparse
Octoparse - . IP- IP-. . , . CSV, Excel API.
ScrapingBee
ScrapingBee - . -, , Chrome.
Luminati
Luminati - - . , .
Scraper API
Scraper API - API -. -, CAPTCHA, HTML- -, API.
Scrapy
- Scrapy. Scrapy - , -. - Python.
Import.io
- -, , . Import.io - CSV. , 1000 API- . Mac OS X, Linus Windows.
BPM RPA
BPM (Business Process Management, -) – , -.
BPM - (, , ..).
BPM- — -.
, , . , , , , - .
- : , . RPA - , .
BPM RPA
Camunda
Camunda , , RPA- , . , Camunda , RPA API / .
:
RPA
RPA
RPA
ELMA
BPM- ELMA . - ELMA — . .
RPA + BPM:
.
.
.
Customer Experience .
ProcessMaker
ProcessMaker - - (BPM) . .
RPA...
RPA
RPA? , . , ( , .), , , , . , .
, :
: . , .
: .
: , . , , , -.
: malware , .
: .
, , , .
:
: - , .
RPA
Let's look from the perspective of a pentester. We can use RPA to create a script that will cyclically check pages for vulnerabilities. Thinking further, we can train an entire army of robots to simulate attacks on various APIs and sites. It is clear that not everything can be automated, but you can try the basic scenarios that are repeated for each site.
To be continued...
In the next article, we will create our own bot, consider what algorithms for detecting bots are, which are written in Selenium and using RPA. Let's create our own algorithm and summarize.