Real-time AI / ML Computing Calls
Let's start with examples from the experience of the Data Science practice of the InterSystems company:
- The “loaded” buyer's portal is connected to an online recommendation system. There is a restructuring of promo-actions on the scale of the retail network (for example, instead of a “flat” line of promo-actions, the “segment-tactics” matrix will now be used). What happens to recommendation engines? What happens to the submission and updating of data to the recommendation mechanism (the amount of input data has increased 25,000 times)? What happens with the development of recommendations (the need to reduce the filtering threshold of recommendation rules by a thousandfold due to a thousandfold increase in their number and "assortment")?
- There is a system for monitoring the likelihood of developing defects in equipment nodes. A process control system was connected to the monitoring system, transmitting thousands of parameters of the technological process every second. What happens to the monitoring system that used to work on “manual samples” (is it capable of providing every second monitoring of probability)? What will happen if a new block of several hundred columns appears in the input data with the readings of sensors recently added to the process control system (will it be necessary and for how long to stop the monitoring system to include data from new sensors in the analysis)?
- A complex of AI / ML-mechanisms (recommendation, monitoring, prognostic), using the results of each other's work, has been created. How many man-hours are required monthly to adapt the operation of this complex to changes in the input data? What is the general “slowdown” with the support of the complex in making management decisions (the frequency of occurrence of new supporting information in it regarding the frequency of occurrence of new input data)?
Summarizing these and many other examples, we have come to the formulation of the challenges that arise in the transition to the use of machine learning and artificial intelligence mechanisms in real time:
- Are we satisfied with the promptness of creation and adaptation (to a changing situation) of AI / ML-developments in our company?
- To what extent do the AI / ML solutions we use support real-time business management?
- Are the AI / ML solutions we use capable of independently (without developers) adapting to changes in data and in business management practices?
Our article is a detailed overview of the capabilities of the InterSystems IRIS platform in terms of universal support for the deployment of AI / ML mechanisms, assembly (integration) of AI / ML solutions and training (testing) of AI / ML solutions on intensive data streams. We will turn to market research, practical examples of AI / ML solutions, and conceptual aspects of what we call a real-time AI / ML platform in this article.
What we know from surveys: real-time applications
The results of a 2019 survey of around 800 IT professionals by Lightbend speaks for themselves:
Figure 1 Leading consumers of real-time data
To quote important snippets of this survey report in our translation:
“… The trends in the popularity of data flow integration tools and, at the same time, support for computing in containers give a synergistic response to the market demand for a faster, more rational, dynamic proposal of effective solutions. Data streams allow information to be transferred faster than traditional packet data. Added to this is the ability to quickly apply computational techniques such as AI / ML-based recommendations, creating a competitive advantage through increased customer satisfaction. The race to agility also affects all roles in the DevOps paradigm — making application development and deployment more efficient. ... Eight hundred and four IT professionals provided information on the use of data streams in their organizations.The respondents were located predominantly in Western countries (41% in Europe and 37% in North America) and were almost evenly distributed among small, medium and large companies. ...
... Artificial intelligence is not a hype. Fifty-eight percent of those already using dataflow processing in productive AI / ML applications confirm that their use in AI / ML will see the largest increase in the next year (compared to other applications).
- Most of those surveyed believed that AI / ML use of data streams would see the biggest gains in the next year.
- Application in AI / ML will grow not only through relatively new types of scenarios, but also through traditional scenarios in which real-time data is increasingly used.
- In addition to AI / ML, the level of enthusiasm among users of IoT data pipelines is impressive - 48% of those who have already integrated IoT data claim that scripting on this data will see a significant increase in the near future. ... "
From this rather interesting survey, it is clear that the perception of machine learning and artificial intelligence scenarios as leaders in the consumption of data streams is already "on the way." But no less important observation is the perception of real-time AI / ML through the optics of DevOps: here we can already start talking about the transformation of the still dominant culture of "disposable AI / ML with a fully accessible dataset."
Real-time AI / ML platform concept
One of the typical real-time AI / ML applications is process control in manufacturing. Using her example and taking into account previous reflections, we will formulate the concept of a real-time AI / ML platform.
The use of artificial intelligence and machine learning in process control has a number of features:
- : ( , )
- , , , , (, )
- , « » , ,
These features force us, in addition to receiving and basic real-time processing of the intense "wideband input" from the process, to perform (in parallel) the application, training and quality control of the results of the work of AI / ML-models - also in real time. The “frame” that our models “see” in the sliding window of relevance is constantly changing - and with it the quality of the results of the work of AI / ML-models trained on one of the “frames” in the past also changes. If the quality of the results of the work of AI / ML-models deteriorates (for example: the value of the classification error "alarm-normal" has gone beyond the boundaries we have defined), additional training of models should be automatically started on a more relevant "frame" - and the choice of the moment to start additional training of models should be the duration of the training itself,and the dynamics of deterioration in the quality of the current version of the models (since the current versions of the models continue to be applied while the models are being trained, and until their "newly trained" versions are formed).
InterSystems IRIS has key platform capabilities to enable AI / ML solutions for real-time process control. These capabilities can be divided into three main groups:
- Continuous Deployment / Delivery (CD) of new or adapted existing AI / ML mechanisms into a productive solution operating in real time on the InterSystems IRIS platform
- Continuous Integration (CI) into a single productive solution of incoming data flows of a technological process, data queues for application / training / quality control of AI / ML mechanisms and exchanges of data / code / control actions with mathematical modeling environments, which are orchestrated in real-time platform InterSystems IRIS
- (-) (Continuous Training, CT) AI/ML-, , (« »), InterSystems IRIS
The classification of platform capabilities in relation to machine learning and artificial intelligence precisely into such groups is not accidental. Let us cite the methodological publication of Google, which provides a conceptual basis for this classification, in our translation:
“… The DevOps concept, which is popular today, covers the development and operation of large-scale information systems. The advantages of implementing this concept are shortening development cycles, accelerating development deployment, and flexible release planning. To reap these benefits, DevOps involves at least two practices:
- Continuous Integration (CI)
- Continuous Delivery (CD)
These practices also apply to AI / ML platforms in order to ensure reliable and efficient build of productive AI / ML solutions.
AI / ML platforms differ from other information systems in the following aspects:
- Team Competencies: When creating an AI / ML solution, the team usually includes data scientists or data science “academics” who conduct data analysis, model development and validation. These team members may or may not be professional developers of productive code.
- : AI/ML- . , , , , . « / », , .
- : AI/ML- , . , .
- : AI/ML- , . AI/ML- , . , - , .
- Productive: AI / ML engines may lack performance not only due to inefficient programming, but also due to the constantly changing nature of the input data. In other words, the performance of AI / ML engines can degrade due to a wider range of reasons than the performance of conventional designs. This leads to the need to monitor (online) the performance of our AI / ML engines, as well as send notifications or discard results if performance indicators do not meet expectations.
AI / ML platforms are similar to other information systems in that they both need continuous code integration with version control, unit testing, integration testing, continuous development deployment. However, in the case of AI / ML, there are a few important differences:
- CI (Continuous Integration, ) – AI/ML-.
- CD (Continuous Delivery/Deployment, ) , , AI/ML-.
- CT (Continuous Training, ) – [. : DevOps, CT , , Continuous Testing], AI/ML-, AI/ML-. …»
We can state that machine learning and artificial intelligence working on real-time data require a wider set of tools and competencies (from code development to the orchestration of mathematical modeling environments), closer integration between all functional and subject areas, more efficient organization of human and machine resources.
Real-time scenario: recognition of the development of defects in feed pumps
Continuing to use the area of technological process control as an example, we will consider a specific problem (we already mentioned at the very beginning): it is required to provide real-time monitoring of the development of defects in pumps based on the flow of values of the technological process parameters and reports of the repair personnel about the detected defects.
Figure 2 Formulation of the task of monitoring the development of defects
A feature of most of the tasks set in this way in practice is that the regularity and efficiency of data receipt (APCS) should be considered against the background of episodic and irregular occurrence (and registration) of defects of various types. In other words: the data from the process control system comes once a second, correct and accurate, and notes are made about defects with a chemical pencil with the date in the general notebook in the shop (for example: "12.01 - flow into the cover from the third bearing side").
Thus, the problem statement can be supplemented with the following important restriction: we have only one “label” of a specific type of defect (ie, an example of a specific type of defect is represented by data from the control system for a specific date - and we have no more examples of this type of defect). This limitation immediately takes us beyond the framework of classical machine learning (supervised learning), for which there should be a lot of "tags".
Figure 3 Clarification of the task of monitoring the development of defects
Can we somehow "multiply" the only "label" we have at our disposal? Yes we can. The current state of the pump is characterized by the degree of similarity to the registered defects. Even without the use of quantitative methods, at the level of visual perception, observing the dynamics of the data values coming from the process control system, you can already learn a lot:
Figure 4 Dynamics of the pump state against the background of the "mark" of a defect of a given type
But visual perception (at least for now) - not the most appropriate "tag" generator in our fast-paced scenario. We will evaluate the similarity of the current state of the pump to the reported defects using a statistical test.
Figure 5 Application of a statistical test to incoming data against the background of a defect "label"
The statistical test determines the probability that the records with the values of the technological process parameters in the “stream-packet” received from the process control system are similar to the records of the “label” of a defect of a certain type. The probability value (the statistical similarity index) calculated as a result of applying the statistical test is converted to a value of 0 or 1, becoming a "label" for machine learning in each specific record in the package being investigated like. That is, after processing the newly received package of pump state records with a statistical test, we have the opportunity (a) to add this package to the training set for training the AI / ML model and (b) to control the quality of the current version of the model when it is applied to this package.
Figure 6 Applying a machine learning model to incoming data against the background of a defect "tag"
In one of our previous webinarswe show and explain how the InterSystems IRIS platform allows you to implement any AI / ML-mechanism in the form of continuously executing business processes that control the reliability of modeling results and adapt model parameters. When implementing a prototype of our scenario with pumps, we use all the InterSystems IRIS functionality presented during the webinar - implementing in the analyzer process as part of our solution not the classic supervised learning, but rather reinforcement learning, which automatically manages the sample for training models. The training sample contains records on which a "consensus of detection" arises after applying both the statistical test and the current version of the model - that is, the statistical test (after transforming the similarity index to 0 or 1),and the model produced the result 1 on such records. With new training of the model, during its validation (the newly trained model is applied to its own training sample, with the preliminary application of the statistical test to it), records that “did not hold” after processing the statistical test result 1 ( due to the constant presence in the training set of records from the initial "label" of the defect), are removed from the training set, and the new version of the model learns from the "label" of the defect plus the "retained" records from the stream.Result 1 “not retained” after being processed by the statistical test (due to the constant presence in the training set of records from the initial “mark” of the defect) are removed from the training set, and the new version of the model learns from the “mark” of the defect plus on the “retained” records from flow.Result 1 “not retained” after being processed by the statistical test (due to the constant presence in the training set of records from the initial “mark” of the defect) are removed from the training set, and the new version of the model learns from the “mark” of the defect plus on the “retained” records from flow.
Figure 7 Robotic AI / ML computing in InterSystems IRIS
In case there is a need for a kind of "second opinion" on the quality of detection obtained during local calculations in InterSystems IRIS, an advisor process is created to perform training and application of models on a control dataset with using cloud services (eg Microsoft Azure, Amazon Web Services, Google Cloud Platform, etc.):
Figure 8 Second opinion from Microsoft Azure orchestrated by InterSystems IRIS
The prototype of our script in InterSystems IRIS is made in the form of an agent-based system of analytical processes that interact with the equipment object (pump), mathematical modeling environments (Python, R and Julia), and provide self-learning of all AI / ML mechanisms involved - on real-time data streams ...
Figure 9 The main functionality of a real-time AI / ML solution in InterSystems IRIS The
practical result of our prototype:
- Defect pattern recognized by the model (January 12):
- A developing defect recognized by the model that was not included in the sample (on September 11, the defect itself was ascertained by the repair team only two days later - on September 13):
Simulation on real data containing several episodes of the same defect has shown that our solution, implemented on the InterSystems IRIS platform, allows us to detect the development of defects of this type several days before they are discovered by the repair team.
InterSystems IRIS is a versatile real-time AI / ML computing platform
The InterSystems IRIS platform simplifies the development, deployment and operation of real-time data solutions. InterSystems IRIS is capable of simultaneously performing transactional and analytical data processing; maintain synchronized data views in accordance with several models (including relational, hierarchical, object and document); act as a platform for integrating a wide range of data sources and individual applications; provide advanced analytics in real time on structured and unstructured data. InterSystems IRIS also provides mechanisms for the use of external analytical tools, allows you to flexibly combine hosting in the cloud and on-premises servers.
Applications built on the InterSystems IRIS platform have been deployed across a variety of industries, helping companies generate significant economic impact from a strategic and operational perspective, increasing decision-making awareness and bridging the gap between event, analysis and action.
Figure 10 InterSystems IRIS Architecture in Real-Time AI / ML Context
Like the previous diagram, the diagram below combines the new "coordinate system" (CD / CI / CT) with the flow of information between platform work items. Imaging begins with the CD macromechanism and continues with the CI and CT macromechanisms.
Figure 11 Diagram of information flows between AI / ML elements of the InterSystems IRIS platform
The essence of the CD mechanism in InterSystems IRIS: platform users (developers of AI / ML solutions) adapt existing and / or create new AI / ML developments using a specialized editor of AI / ML mechanisms program code: Jupyter (full name: Jupyter Notebook; also, for brevity, the documents created in this editor are sometimes called). In Jupyter, a developer has the ability to write, debug, and make sure that a specific AI / ML development works (including using graphics) before it is deployed (“deployed”) to InterSystems IRIS. It is clear that a new development created in this way will receive only basic debugging (since, in particular, Jupyter does not work with real-time data streams) - this is the order of the day,after all, the main result of development in Jupyter is confirmation of the fundamental performance of a separate AI / ML-mechanism ("shows the expected result on a data sample"). Similarly, a mechanism already placed in the platform (see the following macro-mechanisms) before debugging in Jupyter may require a "rollback" to a "pre-platform" view (reading data from files, working with data via xDBC instead of tables, direct interaction with globals - multidimensional data arrays InterSystems IRIS - etc.).working with data via xDBC instead of tables, direct interaction with globals - multidimensional data arrays of InterSystems IRIS - etc.).working with data via xDBC instead of tables, direct interaction with globals - multidimensional data arrays of InterSystems IRIS - etc.).
An important aspect of CD implementation in InterSystems IRIS: a bi-directional integration is implemented between the platform and Jupyter, which allows transferring content in the platform (and, in the future, processing in the platform) content in Python, R and Julia (all three are programming languages in the corresponding leading open- source environments of mathematical modeling). Thus, AI / ML content developers have the ability to "continuously deploy" this content in the platform, working in their usual Jupyter editor, with the familiar libraries available in Python, R, Julia, and performing basic debugging (if necessary) outside the platform ...
Moving on to the CI macro mechanism in InterSystems IRIS. The diagram shows the macroprocess of a "real-time robot" (a complex of data structures, business processes and code fragments orchestrated by them in the Mathred languages and ObjectScript - the native language of InterSystems IRIS development). The task of this macroprocess: to maintain the data queues necessary for the operation of AI / ML mechanisms (based on data streams transmitted to the platform in real time), to make decisions on the sequence of application and the "assortment" of AI / ML mechanisms (they are also "mathematical algorithms", " models ", etc. - can be called differently depending on the implementation specifics and terminological preferences), keep up to date data structures for analyzing the results of AI / ML mechanisms (cubes, tables, multidimensional data arrays, etc.).- for reports, dashboards, etc.).
An important aspect of CI implementation in InterSystems IRIS: a bi-directional integration is implemented between the platform and mathematical modeling environments, which allows executing content placed on the platform in Python, R and Julia in their respective environments with the return of execution results. This integration is implemented both in "terminal mode" (that is, AI / ML content is formulated as ObjectScript code that makes calls to the math medium), and in "business process mode" (that is, AI / ML content is formulated as a business process using a graphical editor, or sometimes using Jupyter, or using IDE - IRIS Studio, Eclipse, Visual Studio Code). The editable availability of business processes in Jupyter is reflected by the link between IRIS at the CI level and Jupyter at the CD level.A more detailed overview of integration with mathematical modeling environments is given below. At this stage, in our opinion, there is every reason to fix the presence in the platform of all the necessary tools for the implementation of “continuous integration” of AI / ML-developments (coming from “continuous deployment”) into AI / ML-solutions in real time.
And the main macro mechanism: CT. Without it, the AI / ML platform will not work (although "real time" will be implemented via CD / CI). The essence of CT is the work of the platform with the “artifacts” of machine learning and artificial intelligence directly in the working sessions of mathematical modeling environments: models, distribution tables, matrix vectors, layers of neural networks, etc. This "work", in most cases, consists in the creation of the mentioned artifacts in the environments (in the case of models, for example, "creation" consists of specifying the specification of the model and the subsequent selection of the values of its parameters - the so-called "training" of the model), their application (for models: the calculation with their help of "model" values of target variables - forecasts, belonging to a category, the probability of an event, etc.) and the improvement of already created and applied artifacts (for example, redefining the set of input variables of the model based on the results of the application - in order to increase the forecasting accuracy, as an option). The key point in understanding the role of CT is its "abstraction" from the realities of CD and CI: CT will implement all artifacts, focusing on the computational and mathematical specifics of an AI / ML solution within the capabilities provided by specific environments. CD and CI will be responsible for “supplying inputs” and “delivering results”.focusing on the computational and mathematical specifics of an AI / ML solution within the capabilities provided by specific environments. CD and CI will be responsible for “supplying inputs” and “delivering results”.focusing on the computational and mathematical specifics of an AI / ML solution within the capabilities provided by specific environments. CD and CI will be responsible for “supplying inputs” and “delivering results”.
An important aspect of CT implementation in InterSystems IRIS: using the already mentioned integration with mathematical modeling environments, the platform has the ability to extract the very artifacts from the work sessions running under its control in the mathematical environment and (most importantly) turn them into platform data objects. For example, a distribution table that was just created in a Python working session can be (without stopping a session in Python) transferred to the platform in the form of, for example, a global (multidimensional data array InterSystems IRIS) - and used for calculations in another AI / ML- mechanism (already implemented in the language of another environment - for example, in R) - or a virtual table. Another example: in parallel with the "normal mode" of the model (in a Python working session), "auto-ML" is performed on its input data:automatic selection of optimal input variables and parameter values. And along with "regular" training, a productive model in real time also receives a "proposal for optimization" of its specification - in which the set of input variables changes, the values of parameters change (no longer as a result of training in Python, but as a result of training an "alternative ”Version of it itself, for example, in the H2O stack), allowing the general AI / ML solution to autonomously cope with unforeseen changes in the nature of the input data and the simulated phenomena.and as a result of training an "alternative" version of it itself, for example, in the H2O stack), allowing the general AI / ML solution to autonomously cope with unforeseen changes in the nature of the input data and the simulated phenomena.and as a result of training an "alternative" version of it itself, for example, in the H2O stack), allowing the general AI / ML solution to autonomously cope with unforeseen changes in the nature of the input data and the simulated phenomena.
Let's get acquainted in more detail with the platform AI / ML functionality of InterSystems IRIS, using the example of a real-life prototype.
In the diagram below, on the left side of the slide, there is a part of the business process that implements the execution of scripts in Python and R. In the central part, there are visual logs of execution of some of these scripts, respectively, in Python and R. Immediately after them are examples of content on one and another language transferred for execution to the appropriate environments. At the end on the right - visualizations based on the results of script execution. The visualizations at the top are made on IRIS Analytics (the data is taken from Python to the InterSystems IRIS data platform and displayed on the dashboard using the platform tools), at the bottom, they are made directly in the R working session and output from there to graphic files. An important aspect: the presented fragment in the prototype is responsible for training the model (classification of equipment states) on data received in real time from the equipment simulator process,on command from the process-monitor of the quality of the classification observed during the application of the model. The implementation of an AI / ML solution in the form of a set of interacting processes (“agents”) will be discussed below.
Figure 12 Interaction with Python, R and Julia in InterSystems IRIS
Platform processes (they are also “business processes”, “analytical processes”, “pipelines”, etc. - depending on the context), first of all, are editable in a graphical editor business processes in the platform itself, and in such a way that both its block diagram and the corresponding AI / ML mechanism (program code) are created. When we say that "an AI / ML mechanism is obtained," we initially mean hybridity (within one process): content in mathematical modeling languages is adjacent to content in SQL (including extensions from IntegratedML), in InterSystems ObjectScript, with other supported languages. Moreover, the platform process gives very ample opportunities for "rendering" in the form of hierarchically nested fragments (as can be seen in the example in the diagram below), which makes it possible to effectively organize even very complex content without ever "falling out" of the graphic format (into "non-graphic »Methods / classes / procedures, etc.). That is, if necessary (and it is foreseen in most projects) absolutely all AI / ML-solutions can be implemented in a graphical self-recommending format. Please note that in the central part of the diagram below, which shows a higher "nesting level", you can see that in addition to the actual work on training the model (using Python and R), an analysis of the so-called ROC-curve of the trained model is added.allowing visually (and computationally too) to assess the quality of training - and this analysis is implemented in the Julia language (executed, respectively, in the Julia framework).
Figure 13 Visual environment for composition of AI / ML solutions in InterSystems IRIS
As mentioned earlier, the initial development and (in some cases) adaptation of AI / ML mechanisms already implemented in the platform will / can be done outside the platform in the Jupyter editor. In the diagram below, we see an example of adapting an existing platform process (the same as in the diagram above) - this is how the fragment of it that is responsible for training the model looks in Jupyter. Python content is available for editing, debugging, graphics output directly in Jupyter. Changes (if necessary) can be made with instant synchronization to the platform process, including its productive version. Similarly, new content can be transferred to the platform (a new platform process is automatically generated).
Figure 14 Using Jupyter Notebook to edit AI / ML engine in InterSystems IRIS
platform Adaptation of the platform process can be performed not only in graphical or notebook format - but also in the “total” IDE (Integrated Development Environment) format. These IDEs are IRIS Studio (native IRIS studio), Visual Studio Code (InterSystems IRIS extension for VSCode) and Eclipse (Atelier plugin). In some cases, it is possible for a development team to use all three IDEs simultaneously. The diagram below shows an example of editing the same process in IRIS studio, in Visual Studio Code and in Eclipse. Absolutely all content is available for editing: Python / R / Julia / SQL, ObjectScript, and the business process.
Figure 15 Developing an InterSystems IRIS business process in various IDEs
The means of describing and executing InterSystems IRIS business processes in the Business Process Language (BPL) deserve special mention. BPL makes it possible to use “ready-made integration components” (activities) in business processes - which, in fact, gives full grounds to assert that “continuous integration” is implemented in InterSystems IRIS. Ready-made components of a business process (activities and connections between them) are the most powerful accelerator for assembling an AI / ML solution. And not only assemblies: thanks to the activities and connections between them, an "autonomous management layer" appears over the disparate AI / ML-developments and mechanisms, capable of making decisions according to the situation, in real time.
Figure 16 Ready components of business processes for continuous integration (CI) on the InterSystems IRIS platform
The concept of agent systems (aka “multi-agent systems”) has a strong position in robotization, and the InterSystems IRIS platform organically supports it through the “product-process” construct. In addition to unlimited possibilities for the "stuffing" of each process with the functionality necessary for a general solution, endowing the system of platform processes with the property of "agency" allows you to create effective solutions for extremely unstable modeled phenomena (behavior of social / biosystems, partially observable technological processes, etc.).
Figure 17 Work of an AI / ML solution in the form of an agent-based system of business processes in InterSystems IRIS
We continue our review of InterSystems IRIS with a story about the applied use of the platform for solving entire classes of real-time problems (a rather detailed acquaintance with some of the best practices of platform AI / ML on InterSystems IRIS occurs in one of our previous webinars ).
Hot on the heels of the previous diagram, below is a more detailed diagram of the agent system. The diagram shows the same prototype, all four agent processes are visible, the relationship between them is schematically drawn: GENERATOR - works out the creation of data by equipment sensors, BUFFER - manages data queues, ANALYZER - performs machine learning itself, MONITOR - controls the quality of machine learning and provides signal about the need to re-train the model.
Figure 18 Composition of an AI / ML solution in the form of an agent-based system of business processes in InterSystems IRIS
The diagram below illustrates the autonomous functioning of another robotic prototype (recognition of the emotional coloring of texts) for some time. In the upper part - the evolution of the model's learning quality indicator (the quality is growing), in the lower part - the dynamics of the model's quality indicator and the facts of repeated training (red bars). As you can see, the solution is efficiently and autonomously self-taught, and operates at the specified quality level (the quality indicator values do not fall below 80%).
Figure 19 Continuous (Self-) Learning (CT) on InterSystems IRIS Platform
We also mentioned "auto-ML" earlier, but the diagram below shows the use of this functionality in detail using another prototype as an example. The graphical diagram of a fragment of a business process shows the activity that launches modeling in the H2O stack, shows the results of this modeling (the apparent dominance of the resulting model over the "man-made" models, according to the comparative diagram of ROC curves, as well as the automated identification of the "most influential variables" available in original dataset). An important point here is the saving of time and expert resources, which is achieved due to "auto-ML": what our platform process does in half a minute (finding and training the optimal model) can take an expert from a week to a month.
Figure 20 Integration of "auto-ML" into an AI / ML solution based on the InterSystems IRIS platform
The diagram below "brings down the climax" a little, but this is a good way to complete the story about the classes of real-time tasks being solved: we remind you that with all the capabilities of the InterSystems platform IRIS, training of models under its control is optional. The platform can receive from the outside the so-called PMML model specification, trained in a tool outside the platform's control - and apply this model in real time from the moment of importing its PMML specification... It is important to take into account that not all AI / ML artifacts can be reduced to the PMML specification, even if most of the most common artifacts allow it. Thus, the InterSystems IRIS platform is "open loop" and does not mean "platform slavery" for users.
Figure 21 Application of the model according to the PMML specification in the InterSystems IRIS platform
Let's list the additional platform advantages of InterSystems IRIS (for clarity, in relation to process control) that are of great importance in the automation of artificial intelligence and real-time machine learning:
- Advanced integration tools with any data sources and consumers (APCS / SCADA, equipment, MRO, ERP, etc.)
- - (Hybrid Transaction/Analytical Processing, HTAP)
- AI/ML- Python, R, Julia
- - (-) AI/ML-
- Business Intelligence AI/ML-
- API AI/ML- /SCADA, - , . .
AI / ML solutions based on InterSystems IRIS platform easily fit into the existing IT infrastructure. The InterSystems IRIS platform provides high reliability of AI / ML solutions by supporting fault-tolerant and disaster-tolerant configurations and flexible deployment in virtual environments, on physical servers, in private and public clouds, and Docker containers.
Thus, InterSystems IRIS is a versatile real-time AI / ML computing platform. The versatility of our platform is confirmed in practice by the absence of de facto restrictions on the complexity of the implemented calculations, the ability of InterSystems IRIS to combine (in real time) the processing of scenarios from a wide variety of industries, the exceptional adaptability of any functions and mechanisms of the platform for specific user needs.
Figure 22 InterSystems IRIS - a universal real-time AI / ML computing platform
For more substantive interaction with those of our readers who are interested in the material presented here, we recommend that you go beyond reading it and continue the dialogue "live". We will be happy to provide support with the formulation of real-time AI / ML scenarios in relation to the specifics of your company, we will perform joint prototyping on the InterSystems IRIS platform, we will form and implement in practice a roadmap for the introduction of artificial intelligence and machine learning into your production and management processes. The contact email address of our AI / ML expert group is MLToolkit@intersystems.com .