I recently saw a post inviting developers to the Flipper Zero project and applied. I was added to the firmware repository and I began to think a lot about the usability of the device. I dreamed that I needed to scan a bunch of keychain keys from different entrances. In the dream, there were different people, apartments, rain, and for some reason I was given a whole handful of these yellow rounds, differing from each other only in the amount of dirt on them. And naturally, in order to somehow cope with this mess, I decided to add all these keys to Flipper Zero.
Right in my dream, I entered new identifiers and they were all added to the menu as "Key", which made the action even more meaningless. The result is a large list, consisting only of the word "Key". I thought I needed to rewrite the firmware. And here is exactly the advantage of sleep - everything conceived comes true at once. Keys began to be added with a random suffix as "Key 74", followed by "Key 22". The number is random, not to be confused with checksums or hashes. In principle, if you have a good memory, you can remember that Key 22 is from Lenin 54, and Key 74 is from Pushkin 29. I thought it would be nice to somehow edit the numbers, right on Flipper, right in the list of keys: we have free left-right buttons, let them increment / decrement the number! A long hold will call the full keyboard! But stopwhat keyboard to draw? Standard Qwerty with switching to Ytsuken? Russified Yaverta? We have a screen of only 128x64 pixels, besides, it is necessary to somehow cram into it the output of what is being typed! How to fit all this? Is it possible to enter text in Morse code?
Primitive keyboard
To say that it is convenient to type anything on such a keyboard is clearly impossible. Which is fast too. Simplicity and a large set of symbols are the only thing that you can get here, but otherwise you want to run from such a keyboard and not look back. Surprisingly, this input method was quite common in the past.
Qwerty
So, what if you take a standard Android keyboard, apply a lens effect and see what happens? 10 hours in After Effects and here it is:
I had a script that converts graphics into a CGA palette, after processing it turned out like this:
I liked it, and even liked the halftones. STOP. Flipper has a black and white screen! But on the other hand, it has energy, which means that you can get semitones by quickly turning on / off pixels. I had to invent my own version of dither. In the process, Smoker Dither was invented:
It is a variation of Baer's dither and a slightly modified Bill Atkinson error distribution matrix. After some debugging and fixing 9000 errors, it turned out like this:
Not to say that I like this option much more, but on the LCD-screen it should look better, half-tones should be especially good due to the constant rotation of the matrices. I haven't received my Flipper yet, so I can't actually test it, but in theory it should look good.
Hmm, what if you try and draw as small a keyboard as possible?
In principle, it worked out well. Everything fits if you want. Optionally, you can nip the RUSLAT key, like many other useful keys. But which ones? Climb into the HID standard and poke the coffee and red button from there?
Layout
The standard QWERTY layout has the following features:
- Designed for typewriters requiring manual input (two hands)
- It is supposed to change hands when typing. While one hand is pressing a key, the other hand prepares to press. However, many words in English can only be typed on the left side of the keyboard.
- The diagonal arrangement of the keys dates back to the physical arrangement of the levers on a typewriter
The following words are attributed to August Dvorak, creator of the “Dvorak keyboard”, an alternative to QWERTY:
, , , , . , , . , , , .But it so happened that neither Dvorak nor his successor, Kolemak, did not gain popularity.
But back to Flipper. Do we have leverage on it? Can I use 2 hands at once to type? Does it make sense to alternate hands? Do we want to use our little fingers more effectively, as the author Kolemak advertises? Do we need compatibility with popular Windows hotkeys like CTRL + C / CTRL + V? The answer to all of these questions is NO. Conclusion: we need to invent our own layout, since QWERTY does not meet our tasks and goals.
Balloons
I took a couple of articles from Wikipedia, copied the text from them and counted the unique characters, here they are: Funny, but there is no double quotation mark ", but there are" Christmas trees. " I tried to visualize the frequency of use of each letter, but I immediately had to apply a logarithmic scale, to see at least something: Let's think about how to arrange these characters on the screen so that it is more convenient for the user to enter them? The very first idea is to calculate the frequency of using this or that letter together with neighbors. And using this data, start the "Balls" simulator, where there is some force of attraction between the balls, equal to the frequency of using these letters in the neighborhood. The first experience was as follows:
!$%&'(),-./01234567
89:;?ABCDEFGHIJKLM
NOPQRSTUVWXYZ[]a
bcdefghijklmnopqrstu
vwxyz|£¥«°²»àéœ́
—’“„…№
↑▲
There is little to see, but if you look closely, you can see how the Cyrillic and Latin alphabets have formed clusters that are practically unrelated to each other, and little-used characters fly out of the space.
After 9000 hours of debugging and pain, a more pleasing simulation was made:
To my disappointment, the symbols are interfering with each other's movement a lot. I tried to play with the force of gravity, changed it dynamically, even made manual "vortices", but nothing helped.
It can already be seen here that although the clusters have already been formed for the most part, they contain foreign elements. We need to come up with some other way of arranging the keys ...
On the other hand, if we put all this in the Voronoi diagram ...
Looks really cool! Hmm, with such a keyboard, you could even leave Flipper and start producing your own real keyboards! Who else needs 1-finger keyboards? What should I write on the kickstarter? I'm going to cry into my pillow over my next useless invention.
Genetics
Okay, after crying, I continued my research. As a basis, I took a prototype of a Chinese keyboard :
Namely, I took a matrix of 16x16 familiarity spaces and put a set of characters in it in random order. In general, at first I wanted to pack them somehow beautifully, using the frequency data on neighboring symbols, analogs of Huffman trees flashed through my head, but I could not give birth to anything sensible. Therefore, I simply placed the keys in the center, placing them in a spiral from the central cell. Subsequently, I started mixing them, so the ordeal with Huffman trees was unnecessary.
And then I just started a simple genetic search: I made a set for mutations, selected the best ones (see below), made them the parents of the next generations. And so until it hangs, i.e. will not run into an evolutionary pit.
There were many experiments:
- 2 , ,
- 30 3000. , , — 30.
- , , .
- , - 5 ,
- , , . - ,
Then I considered how effective this or that mutation was. I had to invent my own grading system. To do this, I took a text from Wikipedia, and made something like a simulator of the typing process. Virtual keyboard 16x16, virtual joystick, virtual text. At the end of the simulation, I looked at how many clicks it took to type this text. This was the efficiency rating. If the number of keystrokes was less than in other variations of the mutation, and even more so before the keys were rearranged, then this “mutation” turned out to be better and it was used to continue the calculation. As a result, it turned out like this:
In some places I really like the resulting numpad, in some places Cyrillic and Latin have formed some semblance of "Yin and Yang", almost everywhere the symbols of the dollar, euro and pounds are next to each other, as well as the percent symbol. The number on the left means how many times you need to press the joystick buttons to type the test text. Perhaps, if I ran the simulator for a week, there would be more interesting options, but I have an old computer for such experiments. And the attentive reader will also find a half-easter-half-bag: almost all layout options contain the word KFC, since it was trained on an article about KFC and some tennis players.
In principle, I am tempted to take and rearrange some keys with my hands, especially the Cyrillic Y, which always turns out to be near the Latin alphabet, or the Latin X, which sometimes happens among the Cyrillic alphabet, and distribute the symbols evenly in a circle or pile them up, but ... And what will evolution answer to this is? How effective would such a rearrangement be? Yes, I was too lazy to write an interactive rearrangement. But it's for the better: now I know that even for the Cyrillic / Latin alphabet there can be several layouts, and let the user choose what is best for him.
If you are interested in other options for layouts that were generated during the calculation, then look at the files anon.fm/articles/flipper-keyboard/layouts.html and anon.fm/articles/flipper-keyboard/layouts-more.html
You can experiment for a long time, here are some ideas for experiments:
- Curling the edges of the space on top of each other. Those. it is not the cursor that moves across the keyboard, but the keyboard moves, and as it moves to one side of the screen, it appears on the other side.
- Rejection of two-dimensional space and wrapping the plane into some kind of three-dimensional object, say, stretching over a globe or some other geometric figure. There are more possibilities for looping space.
- Adding commonly used symbols across multiple instances. For example, why not add a dozen spaces? Here in Kolemak there are 2 backspaces, only in our case it is rather a space to multiply.
- , , . —
- , , Dasher
If anyone does not know about Dasher, then run here: f-droid.org/en/packages/dasher.android - everything about our device and even more here: text input from joysticks, trackballs, mice, eye trackers and other one-handed, one-finger and wearable devices. The interface can be launched even at 32x32 pixels without sacrificing usability. Worked on Pocket PC and survived for years! But, no matter how much I am a fan of this thing, despite the large selection of options, I have not learned how to use it, while feeling the convenience, so for me it is more of a techno-demic of possibilities, the possibility of interacting with a machine. If someone wants to port Dasher, then you are welcome, patches still come to the repository on a regular basis, I think there will be many who wish to see such a thing on their Flipper.
Here you can also add a variety of options for chord keyboards, or on-screen keyboards from the Palm era, such as MessagEase. The idea behind MessagEase is great, but the problem is that holding down 2 buttons is personally unacceptable for me, so personally I will not turn Flipper into a chord keyboard. But I'm sure a real hacking device can't survive without a chord keyboard, so someone else will surely add it.
Summing up: there can be a great variety of keyboard options. But I must not only imagine how it should be, but also implement everything described above. So, if you are willing, you can try, but I have to work.
Implementation
And in the next part we will try to implement everything we have conceived. We will write a small emulator and try to reinvent the API.
Finally, here are some more animations with keyboard concepts:
Keyboard stretched over what Max calls Torus Knot:
And this is an attempt to pull an owl onto a globe ^ W ^ W ^ W keyboard over a sphere. Here I experimented a little with dithering, and I also added Epileptic Dithering to the Dithering of the Smoker.
Similar to the previous one, only for CGA.
UPD: I completely forgot to attach the sources of all this disgrace. Are they interesting to anyone?