The Android code editor: part 2

So the time has come for the publication of the second part, today we will continue to develop our code editor and add auto-completion and error highlighting to it, and also talk about why any code editor EditTextwill not lag.



Before reading further, I strongly recommend that you read the first part .

Introduction

First, let's remember where we left off in the last part . We wrote an optimized syntax highlighting that parses the text in the background and colors only its visible part, as well as added line numbering (albeit without android line breaks, but still).



In this part we will add code completion and error highlighting.

Code completion

First, let's imagine how it should work:

1. User writes a word
2. After entering the first N characters, a window appears with tips
3. When you click on the hint, the word is automatically "printed"
4. The window with hints is closed and the cursor is moved to the end of the word
5. If the user entered the word displayed in the tooltip himself, the window with hints should automatically close

Doesn't it look like anything? Android already has a component with exactly the same logic - MultiAutoCompleteTextViewso PopupWindowwe don't have to write crutches with us (they have already been written for us).



The first step is to change the parent of our class:

class TextProcessor @JvmOverloads constructor(
    context: Context,
    attrs: AttributeSet? = null,
    defStyleAttr: Int = R.attr.autoCompleteTextViewStyle
) : MultiAutoCompleteTextView(context, attrs, defStyleAttr)

Now we need to write ArrayAdapterwhich will display the found results. The complete adapter code will not be available, examples of implementation can be found on the Internet. But I will stop at the moment with filtering.



To ArrayAdapterbe able to understand what hints need to be displayed, we need to override the method getFilter:

override fun getFilter(): Filter {
    return object : Filter() {

        private val suggestions = mutableListOf<String>()

        override fun performFiltering(constraint: CharSequence?): FilterResults {
            // ...
        }

        override fun publishResults(constraint: CharSequence?, results: FilterResults) {
            clear() //    
            addAll(suggestions)
            notifyDataSetChanged()
        }
    }
}

And in the method, performFilteringfill the list suggestionsof words based on the word that the user began to enter (contained in a variable constraint).

Where to get the data before filtering?

It all depends on you - you can use some kind of interpreter to select only valid options, or scan the entire text when you open the file. For simplicity of the example, I will use a ready-made list of auto-completion options:

private val staticSuggestions = mutableListOf(
    "function",
    "return",
    "var",
    "const",
    "let",
    "null"
    ...
)

...

override fun performFiltering(constraint: CharSequence?): FilterResults {
    val filterResults = FilterResults()
    val input = constraint.toString()
    suggestions.clear() //   
    for (suggestion in staticSuggestions) {
        if (suggestion.startsWith(input, ignoreCase = true) && 
            !suggestion.equals(input, ignoreCase = true)) {
            suggestions.add(suggestion)
        }
    }
    filterResults.values = suggestions
    filterResults.count = suggestions.size
    return filterResults
}

The filtering logic here is rather primitive, we go through the entire list and, ignoring the case, compare the beginning of the string.



Installed the adapter, write the text - it does not work. What's wrong? On the first link in Google, we come across an answer that says we forgot to install Tokenizer.

What is Tokenizer for?

In simple terms, it Tokenizerhelps to MultiAutoCompleteTextViewunderstand after which entered character the word input can be considered complete. It also has a ready-made implementation in the form of CommaTokenizerseparating words into commas, which in this case does not suit us.



Well, since CommaTokenizerwe are not satisfied, then we will write our own:





Let's figure it out:

TOKEN - a string with characters that separate one word from another. In the methods findTokenStartand findTokenEndwe go through the text in search of these very separating symbols. The method terminateTokenallows you to return a modified result, but we don't need it, so we just return the text unchanged.



I also prefer to add a 2 character input delay before displaying the list:

textProcessor.threshold = 2

Install, run, write text - it works! But for some reason the window with the tips behaves strangely - it is displayed in full width, its height is small, and in theory it should appear under the cursor, how will we fix it?

Correcting visual flaws

This is where the fun begins, because the API allows us to change not only the size of the window, but also its position.



First, let's decide on the size. In my opinion, the most convenient option would be a window half the height and width of the screen, but since our size Viewchanges depending on the state of the keyboard, we will select the sizes in the method onSizeChanged:

override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
    super.onSizeChanged(w, h, oldw, oldh)
    updateSyntaxHighlighting()
    dropDownWidth = w * 1 / 2
    dropDownHeight = h * 1 / 2
}

It looks better, but not much. We want to achieve that the window appears under the cursor and moves with it during editing.



If everything is quite simple with moving along X - we take the coordinate of the beginning of the letter and set this value to dropDownHorizontalOffset, then choosing the height will be more difficult.



Google about the properties of fonts, you can stumble upon this post . The picture that the author attached clearly shows what properties we can use to calculate the vertical coordinate.



Based on the picture, Baseline is what we need. It is at this level that a window with autocomplete options should appear.



Now let's write a method that we will call when the text changes to onTextChanged:

private fun onPopupChangePosition() {
    val line = layout.getLineForOffset(selectionStart) //   
    val x = layout.getPrimaryHorizontal(selectionStart) //  
    val y = layout.getLineBaseline(line) //   baseline

    val offsetHorizontal = x + gutterWidth //     
    dropDownHorizontalOffset = offsetHorizontal.toInt()

    val offsetVertical = y - scrollY // -scrollY   ""  
    dropDownVerticalOffset = offsetVertical
}

It seems they haven't forgotten anything - the X offset works, but the Y offset is calculated incorrectly. This is because we did not specify dropDownAnchorin the markup:

android:dropDownAnchor="@id/toolbar"

By specifying Toolbarthe quality, dropDownAnchorwe let the widget know that the drop-down list will be displayed below it.



Now, if we start editing the text, everything will work, but over time we will notice that if the window does not fit under the cursor, it is dragged up with a huge indent, which looks ugly. It's time to write a crutch:

val offset = offsetVertical + dropDownHeight
if (offset < getVisibleHeight()) {
    dropDownVerticalOffset = offsetVertical
} else {
    dropDownVerticalOffset = offsetVertical - dropDownHeight
}

...

private fun getVisibleHeight(): Int {
    val rect = Rect()
    getWindowVisibleDisplayFrame(rect)
    return rect.bottom - rect.top
}

We do not need to change the indentation if the sum is offsetVertical + dropDownHeightless than the visible height of the screen, because in this case the window is placed under the cursor. But if it's still more, then we subtract from the indent dropDownHeight- so it will fit over the cursor without a huge indent that the widget itself adds.



PS You can see keyboard blinking on the gif, and to be honest, I don't know how to fix it, so if you have a solution, write.

Highlighting errors

With error highlighting, everything is much simpler than it seems, because we ourselves cannot directly detect syntax errors in the code - we will use a third-party parser library. Since I'm writing an editor for JavaScript, my choice fell on Rhino , a popular JavaScript engine that's time-tested and still supported.

How will we parse?

Launching Rhino is quite a cumbersome operation, so running the parser after each character entered (as we did with highlighting) is not an option at all. To solve this problem, I will use the RxBinding library , and for those who do not want to drag RxJava into the project, you can try similar options.



The operator debouncewill help us achieve what we want, and if you are not familiar with him, I advise you to read this article .

textProcessor.textChangeEvents()
    .skipInitialValue()
    .debounce(1500, TimeUnit.MILLISECONDS)
    .filter { it.text.isNotEmpty() }
    .distinctUntilChanged()
    .observeOn(AndroidSchedulers.mainThread())
    .subscribeBy {
        //    
    }
    .disposeOnFragmentDestroyView()

Now let's write a model that the parser will return to us:

data class ParseResult(val exception: RhinoException?)

I suggest using the following logic: if no errors are found, then there exceptionwill be null. Otherwise, we will get an object RhinoExceptionthat contains all the necessary information - line number, error message, StackTrace, etc.



Well, actually, the parsing itself:

//      !
val context = Context.enter() // org.mozilla.javascript.Context
context.optimizationLevel = -1
context.maximumInterpreterStackDepth = 1
try {
    val scope = context.initStandardObjects()

    context.evaluateString(scope, sourceCode, fileName, 1, null)
    return ParseResult(null)
} catch (e: RhinoException) {
    return ParseResult(e)
} finally {
    Context.exit()
}

Understanding:

The most important thing here is the method evaluateString- it allows you to run the code that we passed as a string sourceCode. The fileNamefile name is indicated in - it will be displayed in errors, unit is the line number to start counting, the last argument is the security domain, but we do not need it, so we set null.

optimizationLevel and maximumInterpreterStackDepth

A parameter optimizationLevelwith a value from 1 to 9 allows you to enable certain code “optimizations” (data flow analysis, type flow analysis, etc.), which will turn a simple syntax error checking into a very time-consuming operation, and we don't need it.



If you use it with a value of 0 , then all these "optimizations" will not be applied, however, if I understand correctly, Rhino will still use some of the resources that are not needed for simple error checking, which means it does not suit us.



There remains only a negative value - by specifying -1 we activate the "interpreter" mode, which is exactly what we need. The documentation says that this is the fastest and most economical way to run Rhino.



The parameter maximumInterpreterStackDepthallows you to limit the number of recursive calls.



Let's imagine what happens if you do not specify this parameter:

1. The user will write the following code:
   
   
   
   

   function recurse() {
       recurse();
   }
   recurse();
   

2. Rhino will run the code, and in a second our application will crash with OutOfMemoryError. The end.

Displaying errors

As I said earlier, as soon as we get the ParseResultcontaining one RhinoException, we will have all the necessary data set to display, including the line number - we just need to call the method lineNumber().



Now let's write the red squiggly line span that I copied to StackOverflow . There is a lot of code, but the logic is simple - draw two short red lines at different angles.





Now you can write a method for installing span on the problem line:

fun setErrorLine(lineNumber: Int) {
    if (lineNumber in 0 until lineCount) {
        val lineStart = layout.getLineStart(lineNumber)
        val lineEnd = layout.getLineEnd(lineNumber)
        text.setSpan(ErrorSpan(), lineStart, lineEnd, Spannable.SPAN_EXCLUSIVE_EXCLUSIVE)
    }
}

It is important to remember that because the result comes with a delay, the user may have time to erase a couple of lines of code, and then it lineNumbermay turn out to be invalid.



Therefore, in order not to get it, IndexOutOfBoundsExceptionwe add a check at the very beginning. Well, then, according to the familiar scheme, we calculate the first and last character of the string, after which we set the span.



The main thing is not to forget to clear the text from the already set spans in afterTextChanged:

fun clearErrorSpans() {
    val spans = text.getSpans<ErrorSpan>(0, text.length)
    for (span in spans) {
        text.removeSpan(span)
    }
}

Why do code editors lag?

In two articles, we wrote a good code editor inheriting from EditTextand MultiAutoCompleteTextView, but we cannot boast of performance when working with large files.



If you open the same TextView.java for 9k + lines of code, then any text editor written according to the same principle as ours will lag.



Q: Why does QuickEdit not lag then?

A: Because under the hood, it uses neither EditText, nor TextView.



Recently, code editors on CustomView are gaining popularity ( here and there , well, or here and there, there are a lot of them). Historically, the TextView has too much redundant logic that code editors don't need. The first things that come to mind are Autofill , Emoji , Compound Drawables , clickable links , etc.



If I understood correctly, the authors of the libraries simply got rid of all this, as a result of which they got a text editor capable of working with files of a million lines without much load on the UI Thread. (Although I may be partially mistaken, I did not understand the source much)



There is another option, but in my opinion less attractive - code editors on WebView ( here and there, there are a lot of them too). I don't like them because the UI on the WebView looks worse than the native one, and they also lose to the editors on the CustomView in terms of performance.

Conclusion

If your task is to write a code editor and reach the top of Google Play, do not waste time and take a ready-made library on CustomView. If you want to get a unique experience, write everything yourself using native widgets.



I will also leave a link to the source code of my code editor on GitHub , there you will find not only those features that I told you about in these two articles, but also many others that were left without attention.



Thank!