The system was often criticized and criticized, mainly due to frequent changes to the API, bugs in the personal account and white spots in some processes.
We decided, albeit not the first, to publish our experience and our thoughts on this new ambitious project of the state, in which we are actively participating.
It may seem that the article is a mess, but the labeling as a whole is such a mess, a lot of aspects converge in one place.
Briefly about us
Cleverens is engaged in software for mobile workers, mainly for special equipment of data collection terminals (TSD) and retirement recorders (RV).
It may seem to someone that there is a forest, a tree in the forest, a beetle on a tree, and a microbe lives on a beetle, and the niche of software for TSD is about microbes on a beetle.
To this we can answer that staff mobility is a very large growing market, and special hardware with ready-made software is generally the future of all current hardware. In the end, any really breakthrough thing is the result of many years of work in a narrow niche ( see Andrey Doronichev's famous interview about mobile YouTube).
Basically, Cleverence products are sold with ready-made integration with accounting systems, which (integration) we ourselves write and support (for example, Warehouse 15 ).
We try to run all software and hardware in real industries, warehouses, shops, etc. For example, when the shoe labeling project was just starting, we went to China to test business processes, and at the same time they shot a video about it.
What is the labeling system and "Fair Mark"
Roughly speaking, this is a public-private Internet of things on barcodes and RFID for total tracking of the movement of units of goods.
There is a lot of information on the labeling system, the main resource is the Honest Sign website .
The attitude of market participants to labeling
It's no secret that the attitude of market participants to labeling is polar. From "Russia, forward!" and “The idea is good, but the implementation let us down” to “they hurt the little ones again”.
The advantages include whitewashing the market and controlling supply chains. Of the minuses - removal from UTII when trading marked goods.
Almost all big business for labeling. Small entrepreneurs who want to work beautifully and in white - also for labeling, for them this is a competitive advantage over gray markets.
Mostly small and medium-sized businesses are against labeling, which considers all this to be an unnecessary burden for themselves.
Claims are mainly formulated as “small business is being strangled again” or “you have so much money and people, you could have done this and that”, “such and such a business process is not well thought out”. Although it was on a small business and its processes that the main money was spent - this is a site with a participant's personal account, a cloud-based code emission recorder, etc.
The position of IT specialists is also worth mentioning, about it below.
At the same time, the information portal of "Honest Mark", on which information is published, is made just fine. For example, where can I find information in English? Right here . And in Chinese? Right here .
Startup problems that everyone was talking about
While the pilot was walking, there were many bugs in the system. Basically, they managed to heal them before July 1, 2020.
They promised that the EGAIS would work better at the start, and in general, the service availability is higher.
Now the system is always available in tobacco and shoes, everything works out quickly, except for a rare shutdown for technological work in the evening for a couple of hours.
But the key problem at the start was that “you can't go without harnessing,” and 99% of the turnover participants (in terms of number) stupidly ignored this system, did not register, did not give a load, waited for the day of judgment, and then demanded a postponement.
2019 «» , 10 , ( ). 10 .
, , .
Everyone knew about this problem, they were preparing for it, but when the shoe deadline came up, it still collapsed. Some UOTs rushed to write out free codes for the remainders for 20 years in advance, this at its peak gave an increase in the load on the servers x40 times. Such games, of course, will not be in vain, because the system is digital and, as they say, “all moves are recorded”.
Unfortunately, the pioneers of marking also suffered - these are the companies that have been in the marking pilot since the first day, tried in good faith, but at the end they also asked for a postponement.
It must be said here that the state system is not an iPhone to cause delight from the first day of sales. And the purpose of its creation was not to collect likes from IT specialists or shoot video reviews like “when it comes out, then I'll see, decide for myself whether to buy”.
For some reason, people expect that from day one the system will work as if it has been in production for 5 years.
In general, if you wait according to this logic, when everything will be perfect, then everything should be postponed to 2030, or better after the conquest of Mars, yeah.
Position of IT specialists
The position of IT specialists is not “for” or “against”, but “give”. Require all changes to be announced in advance, require better documentation, more test cases, sandboxes, fewer restrictions.
In general, the project has a lot of different hardware and international standards, which brings only pain and suffering to classic IT specialists.
IT specialists are very corrosive and intelligent people, with their own opinion on any issue. And there is no consensus on the labeling, except for claims about the API, in general, there is no.
The API of the system changes more often than we would like for a "make and forget" scenario. But at the same time, it is considered that a bunch of methods should be added to the API for working with group packages (aggregates), for simplified authorization, etc., but this again means changing the API.
We are IT specialists ourselves, and we have the same double position that the API lacks methods, but the API changes too often. Here, as they say, "the flag is racist, but this is our story ."
In general, the API needs to be improved, but it is advisable to warn about this 1-2 months in advance and give people time to make changes to their systems. And that was the time when there was a newer version of the API on the tobacco marking combat loop than in the test loop.
Technical aspects of marking
The technical challenges faced by IT pros basically boil down to the following list:
- Sysadmin questions with UKEP, which were and will be as long as there is fragmentation by versions of operating systems and browsers
- Coding issues that cannot be solved by googling on StackOverflow because the system is new and unique despite the use of open source software
- Implementation questions, when people use “online barcode generator” or “keyboard breaker”, and then wonder why nonsense came out
Look, here is Java, and Reactive (RxJava), and Spring and even Netflix:
To form your own opinion, you can read the techno chat in the cart .
Authorization
Authorization in the IS MT takes place in 2 stages.
First, you need to request a certain pseudo-random “key”, which must be signed and sent to the url “/ api / v3 / auth / cert /“ (and at the time of this writing, the slash “/“ must have been at the end of the url, otherwise it did not work, which looks strange).
Obtaining the "key" is as follows (you need to substitute the desired host, there are many different ones, depending on whether it is a sandbox or a combat one, clothes or something else):
// /api/v3/auth/cert/key “” ,
var uri1 = new Uri("< >/api/v3/auth/cert/key")
var randomDataResponse = REST.Invoke<RandomDataResponse>(uri1, "GET", timeout);Then you need to sign this pseudo-random "key" using the selected CEP and send it to the GIS MT in order to receive an authorization token in response.
To do this, we are looking for a CEP certificate for a user-selected fingerprint (certificateThumb). The certificate is in this case in the “Personal” of the local machine. Also, the certificate context must be associated with the key pair store (such certificates are marked with a key icon in the list of certificates in the windows snap-in):
Obtaining a certificate in the code to generate a signature:
X509Certificate2 cert = null;
using(store = new X509Store(StoreName.My, StoreLocation.LocalMachine))
{
store.Open(OpenFlags.ReadOnly);
cert = from c in store.Certificates where string.Compare(c.Thumbprint, certificateThumb, true) == 0 select c;
}We sign in this way:
//
var content = new ContentInfo(randomDataResponse.Data);
var signedCms = new SignedCms(content, false);
// ,
var signer = new CmsSigner(SubjectIdentifierType.IssuerAndSerialNumber, cert);
signer.SignedAttributes.Add(new Pkcs9SigningTime(DateTime.Now));
//
signedCms.ComputeSignature(signer, false);
var sign = signedCms.Encode();A signed CMS message (sign) is the verification signature that needs to be sent for authorization, after being converted to Base64 format.
var tokenRequest = new TokenRequest();
tokenRequest.UUID = randomDataResponse.UUID;
tokenRequest.Data = Convert.ToBase64String(sign);
// /api/v3/auth/cert/
var uri2 = new Uri("< >/api/v3/auth/cert/")
var tokenResponse = REST.Invoke<TokenResponse>(uri2, "POST", tokenRequest);
In response, a JWT token is received , encoded in Base64.
By the way, the documents to be sent are signed in the same way, only the so-called “detached” signature is formed:
var signedCms = new SignedCms(content, true);Helper classes (with a schema identical to the XSD document schemas from the API):
/// <summary>
/// GET /api/v3/auth/cert/key .
/// </summary>
public class RandomDataResponse
{
/// <summary>
/// , string
/// </summary>
[Newtonsoft.Json.JsonProperty(PropertyName = "uuid")]
public string UUID { get; set; }
/// <summary>
/// , string
/// </summary>
[Newtonsoft.Json.JsonProperty(PropertyName = "data")]
public string Data { get; set; }
}
/// <summary>
/// .
/// POST /api/v3/auth/cert/
/// </summary>
public class TokenRequest
{
/// <summary>
/// uuid - /api/v3/auth/cert/key
/// </summary>
[Newtonsoft.Json.JsonProperty(PropertyName = "uuid")]
public string UUID { get; set; }
/// <summary>
/// base64
/// </summary>
[Newtonsoft.Json.JsonProperty(PropertyName = "data")]
public string Data { get; set; }
}
/// <summary>
/// .
/// POST /api/v3/auth/cert/
/// </summary>
public class TokenResponse
{
/// <summary>
/// base64-
/// </summary>
[Newtonsoft.Json.JsonProperty(PropertyName = "token")]
public string EncodedTokenBase64 { get; set; }
}If you managed to log in, you have completed a small quest.
Problems with generating and reading barcodes
The marking system is based on international standards, with the exception of the part with cryptography. This is excellent and commendable.
Unfortunately, the average IT specialist does not read any standards, does not study anything about anything, but immediately sculpts his bicycles from free components about barcode generation, printing, business processes, etc.
The first difficulty is to print the labeling barcode in your design. Out of the box "Honest Sign" provides a free tool, but no one, of course, began to deal with it, because there is an "online barcode generator" and built-in report editors in 1C / SAP / NAV / DAX.
The ambush is that the modern barcode is not just a graphical representation of symbols (we will have a separate article about this). Therefore, the problems of prefixes, separators, encodings, mandatory unprintable characters and so on that no one knew about and did not want to understand immediately surfaced.
Example with GS special character (ASCII 29)
Example with FNC1 special character (ASCII 232)
Screenshots are taken from the Checkmark application .
Here is a piece of the Checkmark source code that parses a barcode scanned by a camera and parsed by our platform:
var km = BarcodeData.GS1;
if (km.Contains("01") && km.Contains("21"))
{
if ((km.Contains("17") || km.Contains("7003")) && !km.Contains("10"))
{
= true;
if ((km.GetValue("01").Length == 14) && (km.GetValue("21").Length == 7 ||
km.GetValue("21").Length == 13) && (km.GetValue("17").Length == 6 ||
km.Getvalue("7003").Length == 10) && (km.GetValue("93").Length == 4))
{
= true;
MSG2 = " KM .";
if (km.GetValue("21").Length == 7)
MSG2 = " .";
}
else
{
// (EntriesRows)
// -
EntriesRows = I(EntriesRows, Al = 93, BarcodeData);
MSG2Error = " 3! .";
Msg = "<b>#N005</b>" + Msg;
}
}
else if (km.Contains("8005") || km.Contains("93"))
{
= true;
if (km.GetValue("01").Length == 14 && km.GetValue("21").Length == 7 &&
km.GetValue("8005").Length == 6 && km.GetValue("93").Length == 4)
{
= true;
MSG2 = " .";
}
else
{
EntriesRows = I(EntriesRows, AI = 8005, BarcodeData)
EntriesRows = I(EntriesRows, Al = 93, BarcodeData)
MSG2Error = " 3! .";
Msg = "<b>#N004</b>" + sg;
}
}
else if (km.Contains("92") && km.Contains("92").Length > 66)
{
= true;
if (km.GetValue("01").Length == 14 && km.GetValue("21").Length == 13 &&
km.GetValue("91").Length == 4 && km.GetValue("92").Length == 88)
{
... ( ..) ...Where did the special characters come from? From the standard. "Honest Sign" did everything according to the standard, and the participants in the turnover of goods also had to read the standard.
BTW: You can only check the correctness of the DataMatrix content by scanning with a camera (even if you scan from a TSD, you need access to the camera). Because all barcode scanners, in accordance with the standard, should not return 1in1 what they read, at least they should throw out the first FNC1 character . In general, scanners can add or remove important barcode characters when scanning, rearrange blocks in places, depending on the settings, all this interferes with verification.
Because Since system integration is not about programs, but about people and how they write and read specifications, the presence of these special characters immediately led to some disagreement when exchanging barcodes.
For example, they could, without really reading the specification, add a string of 4 characters “FNC1” to the barcode instead of one ASCII 232 ( FNC1) character . And yet, when exchanging between subsystems, such characters are often filtered and lost, they reach printing without such characters, while in ordinary text editors special. characters are often not displayed and people wonder why something is wrong if the barcodes match.
Likewise, if you do not specify the symbol FNC1in the barcode, some hardware or software may not understand what it is and how to disassemble it, because this symbol at the beginning of the barcode just indicates that this is a GS1 DataMatrix format.
One recent example that almost everyone stumbled upon was the appearance of parentheses and punctuation marks in the serial number.
Where do the parentheses and punctuation marks come from in the serial number? Out of the standard! The GS1 standard allows them to be used. Because the serial number is only 13 characters long, and there are a lot of goods for marking, you have to use all available characters.
Separate lulzes - brackets in the serial number. They are allowed there by the standard, but no one expected them. In so-called human-readable notation (HID), parentheses have a special meaning: they frame the numbers of meaningful character groups. For example, “(21)” means the serial number will go further down the line.
If it is naive to think that the barcode contains just a string of characters, then in the string “(21) abcd12 (3) d (1, (91)”) the serial number appears to be “abcd12”, and the part about “(1,” seems to be an error at all.
In fact, the serial number here is “abcd12 (3) d (1,”, and after the comma there is an unreadable GS symbol .) And the point is not that the serial number should be 13 characters (because formally the length of the serial in GS1 DataMatrix may be different , and in tires, for example, it can be 20 characters). The point is precisely in the invisibleGS , which must be taken into account, and which is not displayed in the fields of any accounting system, is not accepted for input and is generally prohibited (and it is also not visible in a regular Notepad, it is not displayed, you need Notepad ++).
The presence of such special characters imposes restrictions on the transmission and storage of barcode data; they must be screened or encoded, for example, in Base64.
BTW: there is a standard for storing and transmitting such barcodes as a string (ISO / IEC 15424-2018), but for some reason no one uses it. GS1 DataMatrix barcode data stored as a string must begin with “] d2”.
Now almost everyone has fixed their barcode printing components (probably). But who reads the standards anyway, right?
For verification, homework:Here is the topic, that if the field is of fixed length, then GS does not need to be inserted, and if the length is variable and the value of the field is not of the maximum length, then you need to insert. In fact, this is not how this rule works. Special character GS is not inserted only if AI starts with a pair of digits from this table here:
And in all other cases it is inserted. Does your barcode generation / parsing code work the same?
Another problem is the scanning of the barcode marking with lamer tools such as “scanner in keyboard wedge”. Before labeling, the biggest problem was that an extra zero is being cut or inserted somewhere. With the introduction of new codes, Lulz markings with scanning are taken to a whole new level.
Often these are not problems of the scanner itself or its settings, but problems with a lack of experience and understand the principles of barcoding.
For example, when connecting the scanner "to a keyboard wedge" (or when emulating this mode), as well as when connecting the scanner as a HID device, there may be problems with replacing some characters in a scanned barcode with others or the disappearance of certain characters.
Scheme of connecting the scanner to the keyboard break
When operating in the indicated modes, the scanner does not transmit the characters from the barcode (sic!) To the computer, but the codes of keystrokes on the keyboard, which would have to be pressed in order to type characters from this barcode on the keyboard.
At the same time, it is obvious that several possible characters can correspond to one keyboard key at once, depending on the layout language and pressing the modifier keys (Shift, Caps Lock, Alt and Ctrl).
For example, when you press this key in the lower right part of the keyboard:
The result can be any of the characters '?', '/', ',' Or '.', Depending on the selected layout and the use of modifier keys. What specific character is entered is determined by the operating system keyboard settings, and not by the scanner.
It is because of this that, instead of some serials, completely different ones drop out at the checkout, or the code is not found at all.
By the way, these problems could have been avoided if there was a check sum in the serial, when the last digit can be used to check whether the code is correct, whether it was read / entered correctly. For current accounts, TIN, card numbers, for EAN13, in all, the last digit is a check amount for verification so that people and cars do not make mistakes when entering.
Obviously, here they hoped that DataMatrix is self-checking, plus there is a crypto-tail. But one thing is the whole barcode, self-checking, and another thing is a piece of code with a serial, which could have been entered from the keyboard at all.
The so-called "crypto-tail" caused separate controversies among IT specialists.
"Cryptotail" is a necessary part of the marking barcode, thanks to which it is possible to check with a symmetric key that the serial number of the item is not sucked out of the finger.
The crypto-tail is only in the barcode and is not stored in the marking database so that no hacker can break the base and print other people's barcodes.
Those. this is a kind of digital signature. But since the barcode is not rubber, but there are many goods, the length of this digital signature is short, which does not allow using encryption keys open to everyone.
Of course, all the serial numbers are in the marking database, but if you check it online, then with any drop in the Internet, all warehouses and stores will immediately stand up.
Cryptotail is intended to solve this problem, but so far (for the summer of 2020) it has not started working in full force.
The fact is that cryptography is Russian and is tied to hardware. But there is still no alternative to the Russian crypt, the Russian state system cannot use a foreign crypt, it will be a hole . Therefore, you will have to endure the Russian one, it is reasonable and not difficult.
In general, cryptography is a very complex topic. There is a lot of information and it is very difficult to find a description of how to sign or encrypt data in a particular case.
Everyone who tried to integrate with the labeling API faced this. There were a lot of questions in chats like “Why not log in?”, “How to sign?”, “Share an example!”, “Signature was generated but a refusal came, why?” and many others.
More often than not, the mistakes were commonplace. For example, after the signature, the data was changed, or a space or somehow insignificant character was lost, but this is critical for the signature, and after sending it to the API, a refusal was received without explanation, because the signature has not been verified.
Fortunately, in the chats, they most often helped and answered such questions. In general, all of these problems are being treated or have already been cured by July 1, 2020.
“ ” 1 2020 - , , . , .
At Cleverance, we think that software problems are not the main thing. Worse, the participants in the circulation of goods still do not understand the essence of the changes. This is a much bigger problem than software bugs.
People treat labeling as another accounting report that can be compiled while sitting at a computer in 1C / SAP / NAV / DAX, corrected, changed retroactively, etc. The type will be used by accountants and logisticians, and IT specialists have to "cut" the necessary modules and processing for this.
The main request of our clients is for the warehouse / Chinese / tetyamasha to scan something, and then it “correctly” loaded into 1C / SAP / NAV / DAX. And we are doing it successfully.
But this is an obvious mistake, which will be revealed when companies start balancing the balances with each other, and the state begins to fine.
What is the actual marking system
Product labeling is a system for reflecting real movements.
Real labeling problems
This is purely ours IMHO.
All real problems of an automaton arise from what the marking system really is for, namely, a system for reflecting real movements.
First, the actual accounting of movements contradicts the current principles of accounting for the entire chain, when everything is accounted for by SKU, by batch or by average. Retrospectively, to reduce and correct is everything.
The accounting system in any company is a combed and corrected fairy tale about how great everything turned out after 100,500 calls, corrections, reposting and cleaning up errors.
At one of the extended meetings at a large foreign distributor of drugs, only after an hour of discussion, the chief logistician finally rolled his eyes and said, “Is that a different barcode for each pack?”. And the words “unique”, “serial number”, “each copy”, “scan everything”, etc., apparently flew by.
Second, the actual accounting of movements is contrary to current business practices and principles of separation of concerns.
Even if the company is large and the employee does not have the right to edit some documents - there is a phone number, you can call or write to someone who can correct.
Or, for example, the documents are signed by the director's CEP by some grassroots employee, and if there is two-factor authentication, then via SMS. The labeling system has a block about users and their rights, you can give everyone separate CEPs, but people are in no hurry to do this.
This is a clear problem that will be solved by providing more and more rights and accounting tools to those ordinary employees who are right next to the labeled product.
Moreover, the rules for calculating the cost price or accounting for movements in the accounting system are based on averaging and simplifications such as FIFO / LIFO and the average price. And those accounting principles that have been applied everywhere for decades from production to retail are based on these simplifications and run counter to the real traceability of unique units.
Because of this, in practice, it turns out, for example, that goods are sent to the marking system by FIFO / LIFO, but what was actually shipped is what was closer to the exit.
This is also solved with special equipment and software for ordinary employees who, by the nature of their work, cannot sit at a computer.
Another problem is an attempt to “pull an owl on the globe” and insert marking codes into all standard ERP documents so that an accountant, logistician or responsible manager can see them there.
Although he has nothing to look at them for, he can do nothing with them.
Indeed, well, as an accountant or manager, looking at the monitor, he can understand that he scanned the warehouse there yesterday, and why there are discrepancies highlighted in red on three things of thousands of cigarettes?
This is also solved by tools for grassroots personnel with access to all the necessary information.
Well that is these are not technical problems of some systems, which in general have already been corrected by July 1, 2020, but the issues of restructuring the work of companies on a new track.
conclusions
The main plus of labeling is that the state forced everyone to digitalize and switch to EDF with digital signatures and cryptography.
“The tide raises all boats,” and the transition of the entire market to common rails has a great synergy effect.
Happy automation everyone!