The solution to such problems can be periodic examination of the perimeter of the organization. Network scanners, IoT search engines, vulnerability scanners and security analysis services are suitable for solving the problem. Further in the article, we will consider the types and parameters of scanning, their advantages and disadvantages, tools that are often used, and methods for processing the results.
Ping scan
The first scan to consider is the ping scan. The main task is to detect "live" nodes in the network. Ping scanning refers to the broadcast of ICMP packets. The scanner sends Echo REQUEST packets to the specified IP addresses and expects Echo REPLY packets in response. If a response is received, the host is considered to be present on the network at the specified IP address.
ICMP is widely used by network administrators for diagnostics, therefore, to avoid disclosing information about nodes, it is important to correctly configure perimeter protection. For corporate networks, this type of scan is not relevant for external scanning, because most security tools block ICMP or ICMP responses by default. In the absence of non-standard tasks in the corporate network, the following types of ICMP messages are usually allowed to exit: Destination Unreachable, Echo REQUEST, Bad IP header, and Echo REPLY, Destination Unreachable, Source Quench, Time Exceeded, Bad IP header are allowed to enter. Local networks do not have such a strict security policy, and attackers can use this method when they have already entered the network, but this is easily detected.
Port scan
Let's combine TCP scanning and UDP scanning under the general name - port scanning. Scanning with these methods determines the available ports on the nodes, and then based on the obtained data, an assumption is made about the type of operating system used or the specific application running on the target node. Port scanning refers to test attempts to connect to external hosts. Let's consider the main methods implemented in automated network scanners:
- TCP SYN,
- TCP CONNECT,
- UDP scan.
The TCP SYN method is the most popular, it is used in 95% of cases. This is called a half-open scan because the connection is not fully established. A SYN message is sent to the port under investigation, then a response is expected, based on which the port status is determined. SYN / ACK responses indicate that the port is listening (open), while RST responses indicate that it is not listening.
If, after several requests, no response is received, then the network traffic to the destination port is filtered by means of firewalls (hereinafter we will use the term "port is filtered"). A port is also marked as filtered if an ICMP message is returned with a Destination Unreachable message and specific codes and flags.
The TCP CONNECT method is less popular than TCP SYN, but it is still common in practice. When implementing the TCP CONNECT method, an attempt is made to establish a TCP connection to the desired port with the handshake procedure. The procedure consists in the exchange of messages for negotiating connection parameters, that is, SYN, SYN / ACK, ACK service messages between nodes. The connection is established at the operating system level, so there is a chance that it will be blocked by the protection tool and end up in the event log.
UDP scan is slower and more complex than TCP scan. Due to the specifics of scanning UDP ports, they are often forgotten, because the total time to scan 65,535 UDP ports with standard parameters per node takes up to 18 hours for most automated scanners. This time can be reduced by parallelizing the scanning process and in a number of other ways. Consideration should be given to finding UDP services because UDP services communicate with a large number of infrastructure services that are typically of interest to attackers.
UDP services DNS (53), NTP (123), SNMP (161), VPN (500, 1194, 4500), RDG (3391) are often found on network perimeters. Less common services such as echo (7), discard (9), chargen (19), as well as DAYTIME (13), TFTP (69), SIP (5060), NFS (2049), RPC (111, 137-139) , 761, etc.), DBMS (1434).
An empty UDP header is sent to determine the port status, and if an ICMP Destination Unreachable reachability error with the Destination port unreachable code is returned, this means that the port is closed; other ICMP reachability errors (Destination host unreachable, Destination protocol unreachable, Network administratively prohibited, Host administratively prohibited, Communication administratively prohibited) indicate that the port is being filtered. If the port responds with a UDP packet, then it is open. Due to the specifics of UDP and packet loss, requests are repeated several times, usually three or more. Typically, if no response is received, the port status is determined to be "open" or "filtered" because it is not clear what caused the traffic - blocking traffic by the protection tool or packet loss.
To accurately determine the status of the port and the service itself running on the UDP port, a special payload is used, the presence of which should cause a certain reaction in the application under study.
Rare scanning methods
Methods that are practically not used:
- TCP ACK,
- TCP NULL, FIN, Xmas,
- Lazy Scan.
The direct purpose of the ACK scan method is to identify protection rules and also to identify filtered ports. Only the ACK flag is set in the request packet for this type of scan. Open and closed ports will return an RST packet, since the ports are reachable for ACK packets, but the state is unknown. Ports that do not respond or respond with an ICMP Destination Unreachable message with specific codes are considered filtered.
TCP NULL, FIN, Xmas methods are to send packets with disabled flags in the TCP header. NULL scans do not set any bits, FIN scans set the TCP FIN bit, and Xmas scans set the FIN, PSH, and URG flags. The methods are based on a feature of the RFC 793 specification that when the port is closed, an incoming segment that does not contain an RST will result in an RST being sent in response. When the port is open, there will be no response. An ICMP reachable error means that the port is being filtered. These methods are considered more secretive than SYN scans, but less accurate because not all systems adhere to RFC 793.
Lazy scanning is the most stealthy of the methods, as it uses another node on the network, called a zombie node, to scan. The method is used by intruders for intelligence. The advantage of this scan is that the port status is determined for the zombie host, so using different hosts, you can establish trust relationships between the hosts. A full description of the method is available here .
Vulnerability identification process
By vulnerability we mean the weak point of the node as a whole or of its individual software components, which can be used to implement an attack. In a standard situation, the presence of vulnerabilities is explained by errors in the program code or the library used, as well as by configuration errors.
The vulnerability is filed in MITER CVE and details are published in NVD . A vulnerability is assigned a CVE identifier and an overall CVSS vulnerability score, which reflects the level of risk the vulnerability poses to the end system. For details on assessing vulnerabilities, see our article . The centralized MITER CVE list is a reference point for vulnerability scanners, as the task of a scan is to detect vulnerable software.
A configuration error is also a vulnerability, but such vulnerabilities are rarely found in the MITER database; however, they still end up in the knowledge bases of scanners with internal identifiers. Other types of vulnerabilities that are not in MITER CVE also fall into the knowledge base of scanners, so when choosing a tool for scanning, it is important to pay attention to the expertise of its developer. Vulnerability Scanner will poll nodes and compare the collected information with a vulnerability database or a list of known vulnerabilities. The more information the scanner has, the more accurate the result.
Let's take a look at the scan parameters, types of scans and principles of detecting vulnerabilities using vulnerability scanners.
Scan options
Within a month, the perimeter of the organization can change repeatedly. Performing a forehead perimeter scan can waste time in which the results become irrelevant. With a strong increase in scanning speed, services may "drop". We need to find a balance and choose the right scan parameters. The time spent, the accuracy and relevance of the results depend on the choice. A total of 65,535 TCP ports and the same number of UDP ports can be scanned. In our experience, the average statistical perimeter of a company that falls into the scan pool is two full C-class networks with a mask of 24.
Basic parameters:
- number of ports,
- scanning depth,
- scanning speed,
- parameters for determining vulnerabilities.
By the number of ports, scanning can be divided into three types - scanning the entire list of TCP and UDP ports, scanning the entire list of TCP ports and popular UDP ports, scanning of popular TCP and UDP ports. How to determine the popularity of a port? In the nmap utility, based on statistics collected by the utility developer, the thousand most popular ports are defined in the configuration file. Commercial scanners also come pre-configured with up to 3500 ports.
If the network uses services on non-standard ports, they should also be added to the scanned list. For regular scans, we recommend using the middle option, which scans all TCP ports and popular UDP ports. This option is the most balanced in terms of time and accuracy of results. When conducting penetration testing or full network perimeter audits, it is recommended that you scan all TCP and UDP ports.
An important note: it will not be possible to see the real picture of the perimeter when scanning from the local network, because the firewall rules for traffic from the internal network will apply to the scanner. Perimeter scanning must be performed from one or more external sites; it makes sense to use different sites only if they are located in different countries.
Scan depth refers to the amount of data that is collected about the scan target. This includes the operating system, software versions, information about the cryptography used for various protocols, information about web applications. At the same time, there is a direct relationship: the more we want to know, the longer the scanner will work and collect information about the nodes.
When choosing a speed, it is necessary to be guided by the bandwidth of the channel from which the scanning takes place, the bandwidth of the channel that is being scanned, and the capabilities of the scanner. There are threshold values, exceeding which does not allow guaranteeing the accuracy of the results, maintaining the operability of the scanned nodes and individual services. Do not forget to take into account the time it takes to complete the scan.
Vulnerability Detection Options is the most extensive section of scan options, which determines the speed of scanning and the amount of vulnerabilities that can be detected. For example, banner checks will not take long. Simulations of attacks will be carried out only for certain services and will not take much time either. The longest view is web crawling.
A full scan of hundreds of web applications can take weeks, depending on the vocabularies used and the number of application entry points that need to be checked. It is important to understand that due to the peculiarities of the implementation of web modules and web crawlers, instrumental verification of web vulnerabilities will not give one hundred percent accuracy, but it can greatly slow down the whole process.
It is best to conduct web scans separately from regular scans by carefully choosing which applications to scan. For in-depth analysis, use static and dynamic application analysis tools or penetration testing services. We do not recommend using dangerous checks when performing regular scans, as there is a risk of disrupting services. For details on checks, see the section on the operation of scanners below.
Tools
If you have ever studied the security logs of your sites, you have probably noticed that the Internet is scanned by a large number of researchers, online services, botnets. It makes no sense to describe all the tools in detail, we will list some scanners and services that are used to scan network perimeters and the Internet. Each of the scanning tools serves a different purpose, so when choosing a tool there should be an understanding of why it is being used. It is sometimes correct to use multiple scanners to obtain complete and accurate results.
Network scanners: Masscan , Zmap , nmap... In fact, there are many more utilities for scanning a network, but you hardly need others to scan a perimeter. These utilities solve most of the tasks associated with port and service scanning.
Search engines on the Internet of Things, or online crawlers, are important tools for gathering information about the Internet in general. They provide a summary of site membership, certificates, active services, and other information. It is possible to agree with the developers of this type of scanners to exclude your resources from the scan list or to keep information about resources for corporate use only. The most famous search engines: Shodan , Censys , Fofa .
To solve the problem, it is not necessary to use a complex commercial tool with a large number of checks: it is unnecessary for scanning a couple of "light" applications and services. In such cases, free scanners will suffice. There are a lot of free web crawlers, and it is difficult to select the most effective ones; here the choice is rather a matter of taste; the most famous: Skipfish , Nikto , ZAP , Acunetix , SQLmap .
To perform minimal scanning tasks and ensure "paper" security, budget commercial scanners with a constantly updated knowledge base of vulnerabilities, as well as support and expertise from the vendor, and FSTEC certificates may be suitable. Most famous: XSpider, RedCheck, Scanner-VS.
For careful manual analysis, Burp Suite, Metasploit and OpenVAS tools will be helpful. Google's Tsunami scanner has recently been released .
A separate line worth mentioning is the online vulnerability search engine Vulners... This is a large database of information security content that collects information about vulnerabilities from a large number of sources, which, in addition to standard databases, include vendor security bulletins, bug bounty programs and other thematic resources. The resource provides an API through which you can take results, so you can implement banner checks of your systems without actually scanning here and now. Or use Vulners vulnerability scanner, which will collect information about the operating system, installed packages and check for vulnerabilities through the Vulners API. Some of the resource's functions are paid.
Security analysis tools
All commercial security systems support basic scanning modes, which are described below, integration with various external systems such as SIEM systems, patch management systems, CMBD, ticket systems. Commercial vulnerability analysis systems can send alerts based on different criteria and support different formats and types of reports. All system developers use common vulnerability databases, as well as their own knowledge bases, which are constantly updated based on research.
The main differences between commercial security analysis tools are the supported standards, licenses of government agencies, the number and quality of implemented checks, as well as the focus on one or another sales market, for example, support for scanning domestic software. The article is not intended to provide a qualitative comparison of vulnerability analysis systems. In our opinion, each system has its own advantages and disadvantages. The listed tools are suitable for security analysis, you can use their combinations: Qualys , MaxPatrol 8 , Rapid 7 InsightVM , Tenable SecurityCenter .
How security analysis systems work
Scanning modes are implemented according to three similar principles:
- Audit, or white box mode.
- Compliance, or verification of compliance with technical standards.
- Pentest, or black box mode.
The main interest in perimeter scanning is the black box mode, because it simulates the actions of an external attacker who does not know anything about the scanned nodes. Below is a quick reference for all modes.
Audit is a white box mode that allows you to carry out a complete inventory of the network, detect all software, determine its versions and parameters, and based on this, draw conclusions about the vulnerability of systems at a detailed level, as well as check systems for the use of weak passwords. The scanning process requires a certain degree of integration with the corporate network, in particular, accounts for authorization on the nodes are required.
It is much easier for an authorized user, who is a scanner, to receive detailed information about a node, its software and configuration parameters. During scanning, various mechanisms and transports of operating systems are used to collect data, depending on the specifics of the system from which the data is collected. The list of transports includes but is not limited to WMI, NetBios, LDAP, SSH, Telnet, Oracle, MS SQL, SAP DIAG, SAP RFC, Remote Engine using the appropriate protocols and ports.
Compliance is a mode of checking for compliance with any security standards, requirements or policies. The mode uses mechanisms and transports similar to audit. A feature of this mode is the ability to check corporate systems for compliance with the standards that are embedded in security scanners. Examples of standards are PCI DSS for payment systems and processing, STO BR IBBS for Russian banks, GDPR for compliance with EU requirements. Another example is internal security policies, which may have higher requirements than those specified in the standards. In addition, there are update installation checks and other custom checks.
Pentest is a black box mode in which the scanner has no data other than the target address or domain name. Let's consider the types of checks that are used in the mode:
- banner checks,
- imitation of attacks,
- web checks,
- checking configurations,
- dangerous checks.
Banner checks are based on the fact that the scanner determines the versions of the used software and operating system, and then verifies these versions against the internal vulnerability database. To search for banners and versions, various sources are used, the reliability of which also differs and is taken into account by the internal logic of the scanner. Sources can be service banners, logs, application responses and their parameters and format. When analyzing web servers and applications, information from error and access denied pages is checked, the responses of these servers and applications and other possible sources of information are analyzed. Scanners mark vulnerabilities detected by banner scan as suspected vulnerabilities or as unconfirmed vulnerabilities.
A mock attack is a safe attempt to exploit a vulnerability on a host. Simulated attacks have a low chance of false positives and are thoroughly tested. When the scanner detects a vulnerability signature on the scan target, the vulnerability is exploited. The checks use the methods necessary to detect the vulnerability; for example, an atypical request is sent to an application that does not cause a denial of service, and the presence of a vulnerability is determined by the response typical of the vulnerable application.
Another method: upon successful exploitation of a vulnerability that allows the code to be executed, the scanner can send an outgoing PING or DNS request from the vulnerable host to itself. It is important to understand that it is not always possible to check vulnerabilities safely, therefore, in pentest mode, checks often appear later than in other scanning modes.
Web checks are the most extensive and lengthy type of checks that detected web applications can be subjected to. At the first stage, the web application directories are scanned, parameters and fields are detected where there could be potential vulnerabilities. The speed of such scanning depends on the dictionary used for searching the directories and on the size of the web application.
At the same stage, banners of CMS and application plug-ins are collected, which are used for banner checking for known vulnerabilities. The next stage is the main web checks: search for SQL Injection of various types, search for errors in the authentication and session storage system, search for sensitive data and unprotected configurations, checks for XXE Injection, cross-site scripting, unsafe deserialization, loading arbitrary files, remote code execution and path traversal ... The list can be wider depending on the scanning parameters and scanner capabilities; usually, at maximum parameters, the checks are carried out according to the OWASP Top Ten list .
Configuration checks are aimed at detecting software configuration errors. They identify default passwords or try passwords using a short set of passwords with different accounts. Reveals administrative authentication panels and control interfaces, available printers, weak encryption algorithms, errors in access rights and disclosure of confidential information via standard paths, downloadable backups and other similar errors made by administrators of IT systems and information security systems.
Among the dangerous checks are those, the use of which potentially leads to a violation of the integrity or availability of data. This includes checks for denial of service, SQL Injection options with parameters for deleting data or making changes. Brute-force attacks without limiting brute-force attempts that lead to account blocking. Dangerous checks are rarely used because of the possible consequences, but they are supported by security scanners as a means of emulating the actions of an attacker who will not worry about data safety.
Scans and results
We have reviewed the basic scanning methods and tools, let's move on to the question of how to use this knowledge in practice. First, you need to answer the question of what and how to scan. To answer this question, you need to collect information about external IP addresses and domain names that belong to the organization. In our experience, it is better to separate scan targets into inventory and vulnerability identification.
An inventory scan can be performed much more frequently than a vulnerability scan. In inventory, it is good practice to enrich the results with information about the service administrator, the internal IP address of the service if NAT is used, and the importance of the service and its purpose. In the future, information will help to quickly eliminate incidents related to the detection of unwanted or vulnerable services. Ideally, the company has a process and a policy for placing services on the network perimeter, IT and information security services are involved in the process.
Even with this approach, there is a potential for errors due to human factors and various technical failures that lead to the appearance of unwanted services at the perimeter. A simple example: a rule is written on a Check Point network appliance that broadcasts port 443 from the internal network to the perimeter. The service that was there is outdated and decommissioned. The IT service was not informed about this, so the rule remained. In this case, the perimeter may end up with authentication to the Check Point appliance administration panel or other internal service that was not planned to be hosted there. At the same time, the picture of the perimeter has not formally changed and the port is available.
To detect such changes, it is necessary to scan periodically and apply differential comparison of the results, then there will be a noticeable change in the service banner, which will attract attention and lead to the analysis of the incident.
Elimination of vulnerabilities
The first step to the correct technical implementation of the vulnerability elimination process is to correctly present the scan results that you will have to work with. If several dissimilar scanners are used, it would be most correct to analyze and combine information on nodes in one place. For this, it is recommended to use analytical systems, which will also store all information about the inventory.
The basic way to fix the vulnerability is to install updates. You can also use another method - take the service out of the perimeter (you still need to install security updates).
You can apply compensatory tuning measures, that is, exclude the use of a vulnerable component or application. Another option is to use specialized security tools such as IPS or application firewall. Of course, it is more correct to prevent the appearance of unwanted services on the network perimeter, but this approach is not always possible due to various circumstances, especially business requirements.
Vulnerability elimination priority
The priority of fixing vulnerabilities depends on the internal processes of the organization. When working to eliminate vulnerabilities for the network perimeter, it is important to have a clear understanding of why the service is located at the perimeter, who administers it and who owns it. First of all, you can eliminate vulnerabilities on the nodes that are responsible for the critical business functions of the company. Naturally, such services cannot be removed from the perimeter, but compensatory measures or additional security measures can be applied. With less significant services, it is easier: they can be temporarily removed from the perimeter, slowly updated and returned to service.
Another way is the priority of elimination according to the severity or the number of vulnerabilities on the node. When 10-40 suspicions of vulnerability from banner scan are found on a node, there is no point in checking whether they all exist there, first of all, this is a signal that it is time to update the software on this node. When there is no opportunity for renewal, it is necessary to work out compensatory measures. If the organization has a large number of nodes where vulnerable software components are found for which there are no updates, then it is time to think about switching to software that is still in the update (support) cycle. It is possible that in order to update the software, you first need to update the operating system.
Outcome
All information about services and services on your network perimeter can be obtained not only by you, but also by anyone from the Internet. With a certain accuracy, it is possible to identify system vulnerabilities even without scanning. To reduce the risks of information security incidents, you need to monitor your network perimeter, hide or protect unwanted services in time, and install updates.
It doesn't matter if the process is organized in-house or with the involvement of third-party experts providing services for perimeter control or security analysis. The most important thing is to ensure perimeter control and vulnerability remediation on a regular basis.
Posted by Maxim Fedotov, Senior Specialist, Online Services Department, PT Expert Security Center, Positive Technologies