Clever Geek Handbook

My pet is LinguaPlayer

Today, on Friday, I would like to tell you about one of my pet projects, what interesting things I had to do while working on it and what issues I could not solve for its further development.





So, I had quite a few pet projects of varying degrees of completion. Among them: a social network for writers, a CSS sprite generator, a Telegram bot for dating by interests, and much more. Today we will talk about my latest development.





Like many people these days, I am learning English. I think many people also know that an effective approach in this matter is the maximum immersion in the environment. Phone interface in English, notes in a notebook in English, watching movies in English with English subtitles. Watching a movie in the original, sooner or later there is a need to translate this or that word or phrase that flickers on the screen every few minutes. Without them, nothing is clear at all.





Project idea

So I came up with the idea of ​​a video player with translatable subtitles. The application allows you to translate words and whole phrases while watching a movie. With it, there is no need to switch between applications or pick up a smartphone. Meet LinguaPlayer .





The scheme of work is simple. The user opens the movie file and the subtitle file. Watches the movie as usual. However, now, in addition to the standard hotkeys, he has keys for translating each word separately, translating whole sentences, rewinding from one line to another. There is also translation by hovering the mouse cursor over words or highlighting the desired piece of text. The app is available for Windows and MacOS. All details can be found on the application page .





Technology stack

Electron, . . Chromium, -. – . Visual Studio Code, Skype, Slack. Electron API, JavaScript, . . – , . JavaScript, Angular, jQuery, Vue – .





LinguaPlayer , : TypeScript, React, MobX, Webpack. , : , . . , , . . , DOM . , , , .





, . — srt- . – , .





 node-webvtt. « ». video- «timeupdate», . , «timeupdate» , . .





hash map. (, ), – , . :





{
	//    2 
	5: [1, 2]
	//    3  
	7: [3, 4, 5]
}

0 4 — . , , —  hash map. , . , , . 4 , . , :





//  :  ,    ( ),   , 
class Cue {
  public readonly index: number;
  public readonly startTime: number;
  public readonly endTime: number;
  public readonly text: string;

  constructor(index: number, startTime: number, endTime: number, text: string) {
    this.index = index;
    this.startTime = startTime;
    this.endTime = endTime;
    this.text = text;
  }
}

interface CueIndex {
  //      ( )     ,
  //        
  [key: number]: number[];
}

class SubtitlesTrack {
  private readonly cues: Cue[];
  private index: CueIndex = {};

  constructor(cues: Cue[]) {
    this.cues = cues;

    //       ,  
    this.indexCues();
  }

  private indexCues() {
    this.cues.forEach((cue: Cue) => {
      //               
      const startSecond = Math.floor(cue.startTime / 1000);
      const endSecond = Math.floor(cue.endTime / 1000);

      //   (  )  
      this.addToIndex(startSecond, cue);
      // ,      ,         
      //         
      if (endSecond !== startSecond) {
        this.addToIndex(endSecond, cue);
      }
    });
  }

  private addToIndex(secondNumber: number, cue: Cue): void {
    //       ,     
    if (!this.index[secondNumber]) {
      this.index[secondNumber] = [];
    }

    //         
    this.index[secondNumber].push(cue.index);
  }

  //   
  public findCueForTime(timeInSeconds: number): Cue|null {
    //   timeupdate     
    //     
    const flooredTime = Math.floor(timeInSeconds);
    //      
    const cues = this.index[flooredTime];
    let currentCue = null;

    //      
    if (cues) {
      //   
      for (let index of cues) {
        const cue = this.cues[index];

        //  ,             
        if (this.isCueInTime(timeInSeconds, cue)) {
          //  ,        
          currentCue = cue;
          break;
        }
      }
    }

    //     null,      
    return currentCue;
  }

  public isCueInTime(timeInSeconds: number, cue: Cue): boolean {
    const timeInMilliseconds: number = timeInSeconds * 1000;

    return timeInMilliseconds >= cue.startTime && timeInMilliseconds <= cue.endTime;
  }
}

, , 4 , , 1 4.





 node-sentence-tokenizer. div sentence word , . :





import Tokenizer from 'sentence-tokenizer';

function formatCue(text: string): string {
  const brMark: string = '[br]';
  const tokenizer = new Tokenizer();

  //            
  text = text
    .replace(/\r\n/g, ` ${brMark}`)
    .replace(/\r/g, ` ${brMark}`)
    .replace(/\n/g, ` ${brMark}`);

  //  text    
  tokenizer.setEntry(text);

  //    
  const sentenceTokens: string[] = tokenizer.getSentences();
  //   
  const sentencesHtml: string[] = sentenceTokens.map((sentenceToken: string, index: number) => {
    //    
    const wordTokens: string[] = tokenizer.getTokens(index);
    //    
    const wordsHtml: string[] = wordTokens.map((wordToken: string) => {
      let brTag: string = '';

      //      ,     html   
      if (wordToken.includes(brMark)) {
        wordToken = wordToken.replace(brMark, '');
        brTag = '<br/>';
      }

      //    span   word    br,  
      return `${brTag}<span class="word">${wordToken}</span>`;
    });

    //   ,  ,     span   sentence
    return `<span class="sentence">${wordsHtml.join(' ')}</span>`;
  });

  //     
  const html: string = sentencesHtml.join(' ');

  return html;
}

 Microsoft Translator  , .





,

, MVP, proof of concept. . , -, Urban Dictionary , . ,  LinguaLeo  Skyeng. .  Anki. .





, , . , , . , – Chromium.   , , H.264 FLAC MP3. , . – . , , .





Thus, the main blocking factor right now is content. It should play without any problems in the application, it should be able to get it easily and quickly, and also, it should not violate licenses and copyrights. As soon as the content issue is resolved, I will happily continue to work on the project. In the meantime, if anyone is interested, you can  download and try the  concept version of the application.







More articles:


All Articles