- Snapshots (Backup)
- Tracing
- New features for thin clients
- Cluster operation mode "Read only"
- Running custom code in the sandbox
- Transparent data encryption: master key rotation
- Tools for interrupting user tasks and requests
- Platform-side caching (.NET)
- Connecting client nodes to server nodes through NAT
Snapshots (Backup)
In Ignite 2.9.0, it became possible to create a backup copy of all caches stored on disk (that is, caches operating in Ignite Native Persistence mode ) from the entire cluster. Snapshots can be created online, on an active cluster with a user load. This creates a completely consistent copy of all cluster data.
You can start creating a backup in one of the following ways:
- using a command-line utility control.sh:
control.sh --snapshot create <snapshot name>; - JMX operation:
MBean group="Snapshot", name=SnapshotMXBeanImpl, createSnapshot(<snapshot name>); - the API through the Java:
Ignite.snapshot().createSnapshot("<snapshot name>").
Where
<snapshot name>is the unique name of the snapshot.
After the completion of the snapshot formation in the directory
work/snapshots/<snapshot name>(with the default settings) of each node, the structure of the file storage of this node at the moment of the snapshot start will be recreated. The generated file structure can be used in the future to restore from a backup copy by replacing files with node data with files from the snapshot directory.
You can find more detailed information about working with snapshots in the official documentation .
Tracing
The Ignite monitoring system continues to improve, and one of the significant innovations in the 2.9 release is the tracing subsystem. Tracing allows you to obtain information useful both for debugging at the development stage and for analyzing incidents. With the help of tracing, it became possible to collect distributed low-level information about the progress of various tasks running in the cluster, and use this information to diagnose performance problems. A trace showing the path of a task in the system is formed in the form of a tree, each next level of which gives more detailed information than the previous one.
In Ignite 2.9.0, tracing covers the following internal components:
- Discovery messages;
- Communication messages
- Exchange process;
- transactions.
To view traces, they must be exported to an external system. For these purposes, Ignite uses the OpenCensus library, which out of the box provides several exporters to various systems (for example, in Zipkin).
You can limit the amount of exported information by setting one or more of the above components as the scope and setting the sampling frequency (settings are available for change at runtime).
You can find more detailed information about tracing in the official documentation .
New features for thin clients
The java and .NET thin clients now have Ignite functionality that was previously available only in the thick client.
The ability to use:
- cluster API & cluster group API ( .NET java):
- ;
- ;
- , ;
- ;
- compute API ( .NET java):
- . , p2p class loader , class-path ( );
- Service Grid ( java):
- Ignite. compute API, , .
In addition, the .NET thin client has received the Automatic Server Node Discovery feature, which is enabled in conjunction with the partition awareness functionality. When using "partition awareness", the client establishes a connection not with one server node, but with several at once, in order, if possible, to send a request to the node that is the main one for the data in this request. At the same time, automatic discovery of cluster nodes allows not to list all addresses of cluster nodes in the client configuration. It is enough that the client can connect to at least one live host using the addresses listed in the configuration. The client will receive the addresses of the remaining nodes from the cluster.
For more information on how to use the new features, see the corresponding subsections of the java thin client and .NET thin client documentation .
Cluster operation mode "Read only"
Prior to release 2.9.0, Ignite had only two states of the cluster: the cluster could be either inactive (nodes were collected in a topology, but any actions with caches were prohibited), or active (any actions were allowed). In release 2.9.0, a new cluster state was added - "read-only". It will be useful for performing some maintenance work (such as checking data integrity).
You can find more detailed information about the states of the cluster in the official documentation .
Running custom code in the sandbox
Ignite can run custom code (such as compute tasks, event listeners, various filters) on server nodes. Such code was executed with the same rights as the Ignite system code and the entire java API was available to it without restrictions. Potentially unsafe code could disrupt the cluster's performance (for example, delete Ignite data files, terminate the JVM, etc.).
In version 2.9.0, it became possible to execute such code in the "sandbox" with the rights that were explicitly assigned to the access subject who requested the execution of this code (for example, a client node). The rights assigned to an accessor are a collection of class objects
java.security.Permissionthat are checked by java before performing some action.
For Ignite Sandbox to function, two components must be installed and enabled:
- Java security manager. Responsible for authorizing subjects when making calls to system java libraries. Disabled by default;
- Ignite security processor. Responsible for authenticating access subjects. "Out of the box" with Ignite is not supplied, requires independent implementation and connection using a plugin.
You can find more detailed information about Ignite Sandbox in the official documentation .
Transparent data encryption: master key rotation
Transparent data encryption (TDE) is a functionality that allows you not to store data on disk in clear text. Encrypting data on a disk using a DBMS is required, for example, for PCI DSS data security certification. In Apache Ignite, basic TDE functionality (Phase 1) was implemented in version 2.7. In the current version, the second phase of TDE has been implemented - the rotation of the master key (the cache keys stored on the disk are encrypted with the master key). The third phase of TDE (cache key rotation) will be implemented in the next release.
You can find more detailed information about master key rotation in the official documentation .
Tools for interrupting user tasks and requests
In previous versions of Ignite, there was no consistent mechanism for interrupting user tasks and requests by the administrator. Users had the option to cancel their tasks and requests. For administrators, separate tools were available that did not correlate with each other (for example, it was possible to abort transactions by list, filter, through JMX or the control.sh utility, and "kill" an SQL query using an SQL command
KILL QUERY). In the current release, the administrator has the ability to interrupt
- various types of queries (SQL, scan, continous),
- transactions,
- compute tasks,
- Ignite services,
using a unified interface.
All these kinds of tasks and requests can be interrupted by any of the following methods:
- the control.sh utility;
- via JMX;
- SQL command.
For more information on interrupting user tasks and requests, see the official documentation .
Platform-side caching (.NET)
Ignite.NET adds the ability to use an additional caching layer on the .NET side. Data in .NET memory in this layer is stored in a deserialized form, so you can read the already cached data without additional JNI calls and deserialization. This significantly increases the speed of non-transactional reads.
For more information on platform-side caching, see the official documentation .
Connecting client nodes to server nodes through NAT
In Ignite 2.9.0, a network interaction mode appeared, in which connections between the "fat" client and the server are initiated only on the client side (the server does not initiate connections to the client, but, if direct interaction with the client is necessary, it asks the client to connect to it through the already established client connections to other servers). This mode of operation allows the use of cluster configurations in which NAT is located between the client and server nodes (for example, when clients are running in a virtual environment).
You can find more detailed information about connecting client nodes through NAT in the official documentation .
Conclusion
The above are the most significant changes in the Apache Ignite 2.9.0 release. But the list of changes is not limited to them. As usual, we have fixed many bugs and made many other useful improvements. A complete list of changes can be found in the release notes .