Clever Geek Handbook

My acquaintance with Salyut virtual assistants, as well as with the SmartMarket and SmartApp platforms

Background

In September 2020,  Sberbank  announced a family of virtual assistants:





  • Sber is a male voice. Communicates officially, addresses the user to "you";





  • Athena is a female voice. Communicates officially, addresses the user to "you";





  • Joy is a female voice. Communicates informally, addresses the user to "You";





   SmartApp Studio -  SmartApp   SmartMarket.





SmartApp ( ) — , , : , , , .





 SmartApp Studio   :





  • Graph — . ;





  • Code — SmartApp DSL JavaScript. ;





  • API — ;





24  @smartapp_studio. 3- .





,   :





  • Chat App - . ;





  • Canvas App - . -;





  • Native App - Android-;





  • Provider App - ;





, , Chat App  SmartApp API.

, API .

API, , .





,     (    ), , :





  1. , . , . . , ( ) .   " < >" .   character.





  2. , .





  3. auto_listening -   . .





  4. emotion - . 22 .





  5.  (RUN_APP) (CLOSE_APP) , (ERROR).





  6. (bubble) .





   (App Store, Google Play)  SberBox  SberPortal.

,   ID,  SmartApp Studio. .   .





, - asr_hints.

ASR (Automatic Speech Recognition) :





"asr_hints":{"words":["","","","","","","",""]}

. - !





Chat App

chat app  , anvas app .

  Native App ( ).





, . , , , . "/" ,  12.10.2020 chat app , .





chat app
chat app

,   chat app      .  canvas app, .  canvas app .





Canvas App?

Canvas App – web- ( ),  assistant-client.

Canvas App WebView (Android, iOS). HTML, CSS JS ( frontend) WebView (SberBox, SberPortal,  ).





-   :





:





  • ;





  • webhook (  SmartApp Studio)  ;





  • (  SmartApp Studio — Frontend Endpoint);





  • webhook frontend «»  assistant-client;





  • frontend, webhook, - , - webhook, ;





canvas app ( API)  SmartApp Studio:





 canvas app?

Webhook  frontend  command:





"payload":{
   "items":[
      {
        "command":{
            "type":"smart_app_data",
            "smart_app_data":{"myTest":"data"}
        }
      }
   ]
}

,  smart_app_data,  frontend. , .





 assistant-client:





  •  ReactJS (  )





  • script (  )





 ReactJS  ,  assistant-client  frontend script:





<html>
<head>
    <script crossorigin src="https://unpkg.com/react@17/umd/react.production.min.js"></script>
    <script crossorigin src="https://unpkg.com/react-dom@17/umd/react-dom.production.min.js"></script>
    <script src="https://unpkg.com/@sberdevices/assistant-client@2.6.0/umd/assistant.min.js"></script>
</head>
<body>  canvas app</body>
</html>

canvas app , .

 ReactJS   assistant-client.





  assistant-client  ""  webhook  frontend canvas app,  canvas app  SberBox, SberPortal .Assistant-client  .





body:





    <div id="command"></div>
    <script type="text/javascript">
        var state = { };
        const url = new URL(window.location.href);
        if (url.searchParams.get('devel') === 'true'){
            console.info('Start devel version');
            var token = ''; // <-     smartapp-studio (https://github.com/sberdevices/assistant-client/tree/main/examples/umd-example)
            var initPhrase = '   '; // <-      canvas app
            var ac = assistant.createSmartappDebugger({ getState: () => state, token, initPhrase });
        }else{
            console.info('Start production version');
            var ac = assistant.createAssistant({ getState: () => state });
        }
        var processData = function(command){
            console.info('got command',command);
            document.getElementById('command').innerHTML = JSON.stringify(command);
        }

        ac.on('start', (command) => {
            let initialData = ac.getInitialData();
            for(let i=0; i < initialData.length; i++){
                processData(initialData[i]);
            }
        });
        ac.on('data', (command) => {
            processData(command);
        });
    </script>

assistant.createSmartappDebugger .

URL ( )    devel. : https://example.ru/frontend.html?devel=true





, "" .





webhook smart_app_data  div id "command":





" -" webhook .





ac.on('start',...) . . , smart_app_data  webhook.





ac.on('data',...).





:





,  assistant.createSmartappDebugger    .    assistant.createAssistant.





, webhook  ac.on('data',...)  processData - . (  Audio())  /.





  frontend webhook. , - .

. Sberportal (touch), SberBox .





body :





<button id="next"></button>

script webhook ( ) id :





var el = document.getElementById("next");
el.addEventListener(
   "click",
   function(){
      ac.sendData({ action: { action_id: el.id} });
   },
   false
);

, webhook  SERVER_ACTION:





{
    "messageId":1612645866245327000,
    "sessionId":"130618ec-8cf5-4a64-83e6-29e0dc6deeb1",
    "messageName":"SERVER_ACTION",
    "payload":{
        "server_action":{"action_id":"next"},
        "message":{"original_message_name":"SERVER_ACTION"}
    }
}

webhook webhook frontend .





canvas app  ( localstorage, ).

canvas app  websocket   . canvas app .

.





, ,  github.

,  github pages  frontend github.





canvas chat app webhook .





canvas app
canvas app

, canvas app  .





, canvas app (SberBox, SberPortal ). : 559x568, 768x400, 959x400, 1920x1080.

Chrome   Devices.





canvas app, . SmartMarket     API.





:





  • canvas app (F12);





  • () ;





  • canvas app   ,   ;





  •   ;





  •     ;





  •  Community roadmap  ;





  •  - @smartapp_studio;





I want to thank the SberDevices team   for their openness (especially for the general meetings with the community at zoom) and of course for listening to the opinion of the community. This is definitely your advantage over others.





I also want to thank the users of my smart-up, thanks to whom I was able to win in the  first stage of the competition . Thank you for appreciating my efforts.





I wish you all success!







More articles:


All Articles