Still from the cartoon "Once upon a time there was a dog"
So, standard monitoring tools are becoming insufficient. Several factors contribute to this:
- indicators of compromise (hashes, IP addresses, and domain names) are often one-time use, since changing them is easy for an attacker, especially in the case of APT;
- attackers use legitimate executable files, standard OS tools, etc .;
- to move around the network, in most cases, it is not exploits that are used, but stolen legitimate credentials;
- in practice, malware is often encountered that is not detected by either antivirus or network signatures;
- As the infrastructure grows, it becomes problematic to update signatures and licenses on time.
Even if the attack is successfully detected, you will not have enough information to unequivocally answer the question: the current incident has been exhausted or some other measures are needed. After all, many events are simply not tracked: working with the registry, creating files, loading modules into memory, the command line of the created process, and so on.
Endpoint monitoring significantly expands the ability to detect and prevent attacks: it allows you to go from detecting hashes, IPs and domains to detecting host artifacts, tools and TTP (yes, the very “pyramid of pain”).
A few examples of techniques that are often encountered in practice and are not detected without host monitoring:
- DLL Hijacking
- Living off the land
- Using Mimikatz
Additional endpoint monitoring can be done using built-in OS tools (advanced auditing), free utilities (such as Sysmon), and commercial solutions (EDR class products). Let us consider the advantages and disadvantages of these approaches by examples of detecting various variations of the above techniques.
Advanced audit. Windows Event Logging
Everyone knows about built-in auditing. As practice shows, by including only one collection of process creation events together with the command line, you will greatly facilitate the process of monitoring and investigating incidents (Pareto's law in action).
With proper configuration, we close some gaps in the standard approach and see:
- facts of starting processes together with the command line;
- Decoded PowerShell scripts (Script block logging)
- partially - work with files and the registry;
- activity related to accounts (creating / deleting users, and so on).
We have the opportunity to detect new techniques:
- some DLL hijacking options for creating files with specific paths;
- using LOLBin and Mimikatz by patterns in the command line.
However, an attacker still has the ability to evade detection. In the case of LOLBin, this can be copying a file to another folder under a different name and obfuscating the command line. And Mimikatz can be recompiled with changed commands and lines, which will not allow detection of its use on the command line. Also, options will not be visible when a legitimate binary file subject to DLL hijacking is placed on the machine.
Mimikatz code with modified lines
You can set up Windows auditing on any system without preinstalling third-party software, but there are also significant drawbacks:
1. Inconvenient and poorly scalable setup.
Example: in order to monitor a specific registry branch, you need to separately configure the ACL for it. The very fact that you need to take some action, in practice, translates into problems and time delays, especially in the case of large infrastructures. If this is done to expand monitoring (for example, to detect some way of UAC Bypass), then time is not critical. But if such a need arises during an incident, it complicates the response process.
2. Lack of control.
If events stop registering or entering the monitoring systems used, it will most likely not be possible to understand this in time, since there is no centralization.
3. Ease of opposition.
Even low-skilled attackers know how to hide from standard audits. The only question is how effectively and invisibly they will do it.
Examples of techniques:
- Regular cleaning of logs. The likelihood of successful event recovery depends on how much time has passed since the cleanup.
- Suspending audit service threads and then performing malicious actions or deleting events (for example, using the tool github.com/QAX-A-Team/EventCleaner ).
- Hiding events by breaking the structure of the corresponding .evtx file .
- Temporarily redirecting events to a separate file. We wrote about this technique in the previous article .
4. By itself, setting up an audit will not provide an opportunity to organize monitoring. First, you will have to establish a centralized collection of events, and then use additional tools, for example, SIEM, to analyze them.
5. Low information content of events. There is no way to get the hash of the process being launched, monitor the loading of libraries into the process memory, and so on.
Similarities between Sysmon and EDR
Sysmon is not a complete EDR solution, as it does not have the ability to perform proactive actions on the system. But the mechanism for collecting events is exactly the same as in commercial EDR: it works on two technologies - kernel callbacks and ETW. We will not describe them in detail - we will only consider how events appear and who is the initiator of their creation.
• Kernel callbacks. For example, PcreateProcessNotifyRoutine, PcreateThreadNotifyRoutine.
Where and how these and other callbacks are called can be seen in the corresponding kernel functions. An example of calling callbacks when creating a process is shown below:
The callback loop in CreateProcessW → NtCreateUserProcess → PspInsertThread. That is, calls these callbacks by the parent process thread that called CreateProcess.
• Event Tracing Windows (ETW).
ETW works in a similar way for event occurrences. Let's look at the process creation example again. When CreateProcessW is called, the parent process thread performs the following actions (simplified diagram):
CreateProcessW (kernel32.dll)
NtCreateUserProcess (ntdll.dll, switch to kernel mode)
NtCreateUserProcess (ntoskrnl.exe, then work in kernel mode)
PspInsertThread (callbacks are also called here ' i)
EtwTraceProcess
EtwpPsProvTraceProcess
EtwWrite
Thus, the parent process thread is the direct initiator of the ETW event. The recording of other events, for example, network events, works in approximately the same way. Below is an example of creating events related to network activity:
EtwpNetProvTraceNetwork core function
Two conclusions follow from the above:
- Both Sysmon and EDR use only native Windows capabilities to collect events to ensure reliable operation.
- An attacker could use obfuscation techniques that would apply to both Sysmon and EDR. Kernel callbacks can be dropped in kernel memory using a signed driver. And knowing the described event registration mechanism, you can understand why Sysmon and some EDRs cannot detect certain injection techniques into the process (for example, using APC) or PPID spoofing .
Sysmon
Using Sysmon allows you to expand the capabilities of standard auditing. In this case, events are recorded in a separate log. Some examples of information not found in Windows Auditing, but available in Sysmon:
- more detailed information about executable files being launched (hash, original name, digital signature, and so on);
- loading drivers and libraries;
- changing the status of the Sysmon service;
- creating a thread in another process;
- access to the process;
- hashes of files created in alternate data streams;
- creation of pipes.
The advantages of monitoring the processes described above are obvious (there are many rules and articles on the topic of detecting various techniques in the public domain). Plus, it becomes possible to detect new variations of well-known techniques:
- copying LOLBin under a different name can be detected by the correspondence of the OriginalFileName, Image and Hashes fields from the process creation event;
- you can detect the loading of unsigned libraries, which in some cases allows you to detect DLL hijacking;
- there is the potential to detect Mimikatz using the above methods or by the ProcessAccess event to the lsass.exe process.
Auditing is configured using a configuration file, which in the case of file creation and registry events is much more convenient than ACL configuration.
In this case, the following points must be taken into account:
- The need for additional tools. Since events are logged, then, as in the case of advanced Windows auditing, Sysmon must be used in conjunction with other tools, for example, SIEM.
- . , Sysmon . .
, . - -. , . , Sysmon . : , , .
You can read about ways to bypass some Sysmon features here , here and here .
Endpoint Detection & Response
As infrastructure grows in size and criticality, Sysmon's flaws become significant. High-quality EDRs have several advantages (we will not describe product-specific capabilities):
1) Extended set of logged events
It all depends on the specific product. For example, there is a logging of all interactive input to the command line, which allows detecting techniques that are not visible either with Windows audit or with Sysmon.
It is worth describing the use case for Mimikatz that we saw during one of our investigations. There is an executable file that contains encrypted Mimikatz in resources. When the correct password and initialization vector are passed on the command line, Mimikatz decrypts successfully and accepts commands on the interactive command line. At the same time, no commands appear in the process creation events.
Logging of interactive input, in turn, will help detect such a case if Mimikatz is not recompiled (in our case, the lines were left untouched).
2) Centralized management and configuration
In a large infrastructure, the ability to centrally check the health and change agent settings is critical. Without it, the response can be delayed for hours, or even days.
3) Self-sufficiency
In most cases, EDR is an independent product with its own interface, which can be used not only in conjunction with other tools, but also separately.
It may be possible to write your own detection rules. Since the dataset is larger than that of Advanced Auditing or Sysmon, analysts have more opportunities to detect different attack patterns.
4) Possibility of active response
During an incident response, the need to take action on multiple systems almost always becomes a problem.
EDR allows you to perform many actions in a convenient interactive form:
- work with files and registry
- block network connections
- terminate processes
- scan with yara rules
- collect artifacts for further analysis (for example, a memory image)
- automation of all of the above to one degree or another
- and much more.
Since the EDR product class was originally created to provide host monitoring and response, there are significantly fewer drawbacks to such solutions in this area. It all depends on the capabilities of a particular product. There are also blind spots, for example, there is no in-depth analysis of network activity (a problem that is successfully solved by NTA / NDR products).
Practice shows that the presence of EDR in the infrastructure significantly expands the capabilities to identify threats, as well as speed up the response and facilitate the investigation of incidents, allowing more accurate reconstruction of the chronology of events. Sysmon acts as a half-measure, and in the case of only one standard audit, which is easy to bypass, you have to be content with less informative artifacts, including unusual ones, which we wrote about in the previous article .
Posted by Asker Jamirze, Senior Technical Investigation Engineer, Incident Investigation Department, JSOC CERT