At NORBIT we deal with SRM solutions . Today we will tell you about a special project for our team - the development of the NBT BPMS platform. We did not just create a business solution for the customer, but developed our own product from scratch - all this implies a completely different approach to design, development, team management, organization of change delivery processes and release planning.
In general, the article is not only about the beautiful KDPV. You will also learn:
- about our experience in designing microservice architecture (the choice of tools, approaches to using these tools, namely, abstraction of their use);
- about the development of a designer of business objects and implementation of a designer of business processes in the solution to ensure the approach of Low-code development;
- about how we organized the work on the project and saved the developers from some routine or distracting aspects when working on the system (abstract interservice interactions, code auto-generation, team atmosphere);
- and about which meme helped us in difficult times.
Our division in the NORBIT company is mainly engaged in the automation of procurement processes, for which it creates various kinds of solutions on the .Net platform.
The development of such solutions began with ASP.NET WebForms, then new versions of these solutions were created on the basis of ASP.NET MVC. In addition to development on .NET, we have had and still have other projects and solutions, but nevertheless, development on .NET accounts for about 80% of all projects of the department.
The limitations of both ASP.NET WebForms and ASP.NET MVC suggest that for the solutions to function on these platforms, deployment on an OS of the Windows Server family was required, and on the DB (database) side, a volume of logic was implemented that did not allow for a quick and painless transition on a DBMS other than MS SQL Server. This did not provide for any obstacles and difficulties before the start of the massive transition to domestic software.
Starting from 2014-2015, the IT market began to move very seriously towards import substitution, and the issue of deploying our systems on operating systems and DBMSs that would fit the new requirements was quite acute. In fact, it became question number 1which we needed to solve so that our solutions could cover the requirements of potential customers in the long term.
At the same time, having strong competencies and a fairly large team of .NET developers, it did not seem rational to start developing a new cross-platform kernel other than .NET. The release of the open source .NET Core platform was very helpful for us, including because of its compatibility with operating systems such as Windows, Linux and macOS.
Question number 2 , which we needed to solve, was that most of the previously created solutions assumed mandatory programming to add attributes of business objects or change business processes in the system.
It is associated with two aspects.
- ( ) - , , , . , , - , , -.
- Low-code development, , , « » - -.
And question number 3 , which we have been facing for many years, is the imperfection of the architectures of the systems being created, which does not allow us to quickly rewrite some part without the need to rewrite individual modules, or creates a sufficiently high entry threshold for programmers, analysts and testers when connecting to projects.
There were many more questions and problems at the entrance to the project, which will be discussed later. But these three (import substitution and open-source, Low-code, low entry threshold and fast immersion rate of new team members) we consider the key ones that we had to solve.
Design
Thinking about these questions led us to the realization that we need a kind of constructor that analysts can “play” with and design a system (business objects and business processes) according to the needs of the customer. It sounds daring, but it's much more interesting than writing another system for a customer. In fact, we decided to make our product in the form of a constructor and develop it.
We definitely did not come up with anything revolutionary, and there are many similar products on the market, but it was fundamentally important for us to solve our accumulated questions, especially since, for the reasons indicated earlier, it was difficult for us, to put it mildly, to follow the path of refactoring existing systems.
At the design stage, we had to rethink the fact that we do not always know in advance who our potential customer is, big or small, demanding or loyal, what business processes he has, what kind of infrastructure. Some of the requirements for the future system began to emerge on their own: you need scaling, you need cross-platform, you need flexible and quick customization of business objects / business processes, an interface, probably a "wagon and a small cart" of integrations. It became scary, very scary.
But, as one of our team leaders says, "The elephant needs to be eaten bit by bit, starting from the foot." At this stage, we set ourselves two tasks: the choice of microservice architecture and the choice of tools. Yes, just microservice right away (Martin Fowler recommends MonolithFirst , but we decided to disobey him), because with monoliths we have long been on "you" and it's time to accept the challenge to move on. But seriously, the requirement of horizontal scaling of the system alone is quite enough, in our opinion.
Choice of architecture and tools
When designing the architecture, we pursued several goals:
- abstraction of the tools used in order to be able to replace an unsuitable tool with another at any time;
- ( , );
- , .
Since our team specializes in Microsoft technologies, .NET Core version 2.1 was chosen as the implementation platform. At the time when the development of our product began, this version was LTS (long term support), and for us this was an important criterion, since some domestic operating systems (on which we could potentially deploy the system) only support .NET Core LTS versions ...
They decided to make Frontend in React + TypeScript, since it is easy to learn. We decided to promote the full stack approach of developers.
We decided to make interservice communication synchronous, because our call chains were supposed to be short, and interservice communication is still abstracted. A service calls another service through an abstract proxy. And it can contain any logic. In our case, this is a call to another service via http via Service Discovery. This design allowed us, on the one hand, to simplify the life of developers (they can simply use the service interface to call it, but in fact, in the ASP.NET Core service container, the registered implementation of this interface is the same Proxy), and on the other - to provide horizontal scaling of the system automatically, since we always ask Service Discovery how to call a service, and that, in turn, can give one of the running instances of this service.
At some point, the service structure became standard, and we made extensions for Visual Studio, which, when a new project is added to the solution, generates a project structure with a minimal service implementation, again, so that developers do not have to do this routine. In addition to this, we made several scripts to launch the entire system with a slight movement of the hand (later we thought about the same quick deployment of a lightweight orchestrator from the developer).
The selection process was also quite interesting. We had to decide which functionality we write ourselves and which we take "out of the box" (talking about existing tools), but, of course, abstracting all interaction with it, so that, if necessary, it would be possible to replace this tool or library with another implementation. We needed the following tools: search engine, business process engine (BPMN Engine), service discovery protocol (Service Discovery), scheduler. The criteria were different: licenses, speed of work, speed of immersion of the project team, etc. The result is the following list: Elasticsearch, Camunda, Consul, Hangfire.
We immediately laid down support for at least PostgreSQL and SQL Server, but focused mainly on PostgreSQL and deployment under Linux. Free software, you know. In this regard, a curious story happened. We laid down the support for SQL Server at the early stages, but did not really expect that some of our potential customers would be interested in deploying on Windows + SQL Server, so we postponed testing this configuration until later (after all, it is always known in advance what environment the customer has ?!) ... And then one day, making another test stand for a potential customer, we were already ready to deploy AltLinux + PostgreSQL as usual, but suddenly we find out that the customer nevertheless changed his mind and decided to deploy on Windows + SQL Server, and we have two days to do everything prepare. Fortunately, the long-written and forgotten code came in handy,who provided this support to us with minor improvements, and we managed to prepare everything on time, and the system worked as if nothing had happened (glory to .NET Core).
Implementation of business ideas
We approached the implementation with a prepared list of stages of work, and the most important of them was getting MVP (minimum viable product) in the near future. Almost from the very first iterations, we began to work with an emphasis on getting a prototype with a visual part as quickly as possible, which can be demonstrated to customers (who were our leaders in the beginning) periodically. This approach, as expected, contributed to the rethinking of some of the previously conceived concepts. I managed to catch several contradictions and fight them in advance. It's about designing a Web API for frontend and backend communication.
Business object constructor
It was already mentioned above that one of the key ideas of the product is the ability to create a business object structure by a configurator user. We actually started with the implementation of the constructor of business objects. The first versions, of course, were more conceptual, but they subsequently gave us rich food for thought. The once unpretentious business objects with a couple of simple fields have subsequently evolved into multi-level multifunctional units with a wide range of all kinds of settings. We learned how to configure not only simple fields, but also fields with a choice from hierarchical directories, related collections of objects with filtering, and so on.
The business object generated by the configurator's mouse has enough metadata to be able to collect custom searchable registries and different presentation forms.
The issue of validating business objects has not spared us either. We quickly discovered that validation cannot but be accompanied by the business logic that is required at the stage of filling the business object with data. “If“ field 1 ”has“ value 1 ”, then“ field 2 ”needs to be hidden” or “Field 1 is the sum of field 2 and field 3” are just a couple of the simplest examples that can be given. How long is it short, and we have an event engine in which this behavior can be configured. Events can also be configured with the mouse. Many options have already been implemented, including complex calculations. But this is a topic for another conversation. Please write in the comments if you solved a similar problem, it would be interesting to exchange feelings.
Business Process Designer
As we like to repeat: "Who needs business objects without business processes?"
results
A year and a half has passed since the start of the project. The team has increased tenfold, the
The first thing that seems mega-cool is the fact that the more functionality we make, the more ideas we get. It feels like we are moving not from the beginning of the project to its completion, but simply from the beginning and further - this is an important aspect. After many years of supporting ready-made solutions (and this happens very often in the life of an IT specialist), this state of affairs is very motivating.
Secondmega-sensation is to watch such a motivated team and be inside it. The process of creating a new product is not only a sequence of tasks and stages. This is, first of all, the atmosphere. The atmosphere of the fact that you very often do things that you have never done before, although you have participated in dozens of projects. Each member of the team breathes it and at some point realizes that he left the comfort zone after the first breath. And the most difficult and main task is to make this atmosphere fresh, comfortable and interesting. We cannot hide from bugs, deadlines, etc., but the lack of a supportive environment in which we all find ourselves can be much more destructive.
We try to pay special attention to the development process of our product. This process is not static. He undergoes changes, if necessary, but he makes it possible to comfortably solve his problems and help his colleagues do it.
PS It is difficult to describe everything in detail in one article, so it turned out to be more of an overview. We really hope that you found something interesting in it, and we will be happy to answer your questions in the comments, or describe some aspect of our system in more detail in a separate article.