Release v0.3.13-beta10

* feat: Add Japanaese JIS layout

* feat: Add KanaSelector and KanaSizeSelector

* test: Try out new selectors

* fix: Add new selectors to polymorphic key data list

This should help make florisboard recognize the new selectors
when used in layouts.

* fix: Implement `evaluteKana` for ComputingEvaluator

This should fix the crashes, hopefully

* feat: Rework the JIS layout

* fix: Implement basic logic for kana state

* feat: Add popups for JIS

* chore: Rename JIS specific layouts

For some reason this change broke the pop-ups

* feat: Add Kana Unicode compositor

* feat: Implement flipping small state and refactor

This should allow small kana characters to be switched over to their
normal counterparts. Also clean up and refactor `KanaUnicode` and make
the filename more appropriate

* Merge `master` to `japanese`

* chore: Add internal codes for Kana switching

This should be the starting point for getting actual keys
to switch between the kana types

* feat: Add initial modifer layout for JIS

* chore: Add keycode for ideographic CJK space

* feat: Add logic for kana switching

* chore: Update layout modifer to use kana switch

* feat: Add icon for kana switcher button

* fix: Scale the icon and update relevant entries

* feat: Begin work on kana selector popup

* Add kana attributes to keyboard state

* Add workaround for Japanese popups

* chore: Remove URL popups for JIS layout

* chore: Add codes and groups for kana switching

This commit adds the "~kana" group with groupId 4.

* feat: Use "~kana" in JIS layout.

* feat: Remove `KanaSizeSelector`.

It will be redundant when compositor handles it instead.

* feat: Add alpha code for making kana small.

This code currently uses a sentinel character to trigger
the transformation. This is not ideal as that means that users
won't be able to use that character in the keyboard. This won't
be a problem if the sentinel character is a Unicode codepoint
that cannot or would not be used normally. As such, this is just
a demonstration of the how the feature should function not it's
final implementation.

* test: Add kana small key in JIS for testing.

This is to preview changes to the kana small algorithm,
This commit should be removed from history later on.

* fix: Add `KANA_SWITCHER` to exceptionsForKeyCodes

* chore: Change "~kana" groupId to avoid conflicts

* fix: Change Kana small behaviour

* chore: Change internal kana small key code

* test: Remove kana small button from JIS layout

* feat: Add iteration marks to daku list

* feat: Allow swtiching between kana modifiers

This allows users to add/remove dakuten/handakuten/small
from kana and swtiching between the three. This is helpful
as it allows users to quickly rectify mistakes. It also lays
the groundwork for the future 12 flick layout

* feat: Change Kana swticher behaviour on katakana

Makes katakana switch over to hiragana immediately. As half-width
katakana is not usually used. The kana-switcher pop-up suffices
switching over to it when needed.

A potential downside to doing it like this making the half-width
katakana more hidden and less likely to be used.

* Add CJK_SPACE special behavior to input logic

* feat: Add CJK punctuation to popups and JIS layout

* feat: Add CJK punctuation symbols

* feat: Add CJK row numbers

* feat: Add postal mark symbol in CJK symbols

And change kana small internal code. Used to be set to
postal mark with face. Now set to geta mark.

* chore: Set default numeric row to CJK for JIS

* feat: Add modifiers layouts for CJK punctuation

* chore: Add latin half-width braces to CJK layout

* fix: Change composer behaviour on empty string

* chore: Add non-composing (han)daku to JIS popups

* fix: Change composer behaviour to be consistent

Also clean up the code a little.

* fix: Change composer empty string behaviour

* feat: Add WidthSelector and migrate half_kata Kana

This adds width selector for all languages instead of tying it down
to only `KanaSelector`. As such `KanaSelector`'s `half_kata`' field
is obsolete.

* fix: Fix bug in composing logic

* fix: Implement width properly and remove half_kata

* refactor: Clean up JIS layout JSON

* fix: Change JIS punctuation to use width_selector

* Improve half width implementation / Clean-up code base

* fix: Remove Half-kata from kana switcher cycle.

* feat: Add char width keycodes and implementation

* feat: Add half-width forms to symbols/cjk layouts

* feat: Add char width switcher to CJK mod layouts

Icons are yet to be added, so the labels will be strings for now

* feat: Add half-width forms for symbols2/cjk layout

* chore: Add more appropriate CJK chars to symbols2

* feat: Add icons for character width switching.

Note that `ic_keyboard_char_width_full` and `ic_keyboard_char_width_half`
are just copies of `ic_keyboard_char_width_switcher` for now.

* feat: Add relevant entries to show char width icons

* chore: Update the character width pop up icons.

And rename the internal names so that they match up with kana_switcher.

* fix: Make char width swticher icon toggle instead

* fix: Make character width icons more consistent.

* chore: Remove reference to char_width_switcher icon

* test: Iest code for why the composer is not working

* test: Remove test code for KanaUnicode

This reverts commit f6ae545183.

* fix: Correct small kana association for E and U

* test: Remove test code for KanaUnicode

This reverts commit f6ae545183.

* fix: Don't add small sentinel on non small char.

* fix: Correct small ya entry in KanaUnicode

* feat: Make kana switcher icon toggle between modes

* fix: Update svg colors to fit with other vectors

* Update app/src/main/java/dev/patrickgold/florisboard/ime/text/TextInputManager.kt

Co-authored-by: Patrick Goldinger <patrick.goldinger@pm.me>

* chore: Fix author name

* chore: Fix author name in CJK numbers

* chore: Fix indentation on author field

Co-authored-by: Waelwindows <waelwindows@hotmail.com>
Co-authored-by: Patrick Goldinger <patrick@patrickgold.dev>

.github/ISSUE_TEMPLATE/feature_request.md

@@ -8,12 +8,11 @@ assignees: ''
 ---
 
 <!--
-- Describe your idea in a short but concise way.
-- If you have multiple ideas which are not directly connected to each
-  other, file an issue per idea. This makes it easy to implement one
-  feature proposal at a time.
-- If you have any examples, e.g. screenshots or other keyboards which
-  have the proposed feature implemented, link them here.
-- Please search existing proposals to avoid creating duplicates.
-- Thank you for your help in making FlorisBoard better!
+Thank you for your help in making FlorisBoard better!
+
+Guide to a good feature-request:
+• Please search existing proposals to avoid creating duplicates.
+• If you have multiple ideas which are not directly connected to other, file a new issue for each idea. This makes it easier to implement your proposals.
+• Describe your idea in a short but concise way.
+• If you have any examples, e.g. screenshots or other keyboards have the proposed feature implemented, feel free to post them after your description.
 -->

.github/ISSUE_TEMPLATE/question.md

@@ -8,9 +8,8 @@ assignees: ''
 ---
 
 <!--
-- If you need assistance in using FlorisBoard, ask it here!
-- If you want to suggest an idea for this project, please use the
-  Feature request template instead.
-- Please search existing questions to avoid creating duplicates.
-- Thank you for your help in making FlorisBoard better!
+If you need assistance in using FlorisBoard, ask it here!
+
+• Please search existing questions to avoid creating duplicates.
+• If you want to suggest an idea for this project, please use the Feature request / Suggestion template instead.
 -->

.github/workflows/android.yml

@@ -11,11 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v1
-      - name: set up JDK 1.8
+      - uses: actions/checkout@v2
+      - name: Checkout submodules
+        run: git submodule update --init --recursive
+      - name: set up JDK 11
         uses: actions/setup-java@v1
         with:
-          java-version: 1.8
+          java-version: 11
       - name: Setup CMake and Ninja
         uses: lukka/get-cmake@v3.20.1
       - uses: actions/cache@v2

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "app/src/main/icu4c"]
+	path = app/src/main/icu4c
+	url = https://github.com/florisboard/icu4c

CONTRIBUTING.md

@@ -41,7 +41,7 @@ syntax (it is very easy though by just looking at some other layout files).
 There are two main steps in adding new layouts, though the config step can
 be skipped if you only add a layout without a new default language support.
 
-### The config file (`app/src/main/assets/ime/config.json`)
+### The config file ([`app/src/main/assets/ime/config.json`](app/src/main/assets/ime/config.json))
 
 This file is very important, as it defines all default currency sets as
 well as all default subtypes available in the Settings Subtype UI. Note
@@ -66,7 +66,7 @@ pre-configured language.
 
 Since v0.3.10-beta05 it is possible to add custom layouts for all types.
 
-To add a new layout, head to `app/src/main/assets/ime/text` and then select
+To add a new layout, head to [`app/src/main/assets/ime/text`](app/src/main/assets/ime/text) and then select
 the correct sub-directory for the type of layout you want to add. In most cases
 this will be `characters` to add a layout like QWERTY etc.
 
@@ -74,14 +74,14 @@ For the `code` field of each key, make sure to use the UTF-8 code. An
 useful tool for finding the correct code is [unicode-table.com](https://unicode-table.com/en/).
 From there, you search for your letter and then use the HTML code, but without the `&#;`
 For internal codes of functional or UI keys, see
-`app/src/main/java/dev/patrickgold/florisboard/ime/text/key/KeyCode.kt`.
+[`app/src/main/java/dev/patrickgold/florisboard/ime/text/key/KeyCode.kt`](app/src/main/java/dev/patrickgold/florisboard/ime/text/key/KeyCode.kt).
 
 The label is equally important and should always match up with the defined
 code. If `code` and `label` don't match up, FlorisBoard won't crash but
 it will most likely lead to confusion in the key processing logic.
 
 Any accents or diacritics that should be exposed via long press can be
-added at `assets/ime/text/characters/extended_popups/<languageTag_name_here>.json`.
+added at [`app/src/main/assets/ime/text/characters/extended_popups/<languageTag_name_here>.json`](app/src/main/assets/ime/text/characters/extended_popups).
 For each key, you can add 1 main and several relevant accents. The main
 accent should be used for accents which are important for the language
 you add. The main field is used for determining if a hint or an accent

README.md

@@ -45,7 +45,11 @@ _A. IzzySoft's repo for F-Droid_:
 
 [<img src="https://gitlab.com/IzzyOnDroid/repo/-/raw/master/assets/IzzyOnDroid.png" height="64" alt="IzzySoft repo badge">](https://apt.izzysoft.de/fdroid/index/apk/dev.patrickgold.florisboard.beta)
 
-_B. Use the APK provided in the release section of this repo_
+_B. Google Play_:
+
+Follow the same steps as for the stable track, the app can then be accessed [here](https://play.google.com/store/apps/details?id=dev.patrickgold.florisboard.beta).
+
+_C. Use the APK provided in the release section of this repo_
 
 ### Giving feedback
 If you want to give feedback to FlorisBoard, there are several ways to
@@ -113,67 +117,7 @@ milestones, please refer to the [Feature roadmap](#feature-roadmap).
 * [ ] ...
 
 ## Feature roadmap
-This section describes the features which are planned to be implemented
-in FlorisBoard for the next major versions, modularized into sections.
-Please note that the milestone due dates are only raw estimates and will
-most likely be delayed back, even though I'm eager to stick to these as
-close as possible.
-
-### [v0.4.0](https://github.com/florisboard/florisboard/milestone/4)
-- Module A: Smartbar rework (Implemented with [#91])
-  - Ability to enable/disable Smartbar (features below thus only work if
-    Smartbar is enabled)
-  - Dynamic switching between clipboard tools and word suggestions
-  - Ability to show both the number row and word suggestions at once
-  - Better icons in quick actions
-  - Complete rework of the Smartbar code base and the Smartbar layout
-    definition in XML
-
-- Module B: Composing suggestions (Phase 1: [#329])
-  - Auto-suggestion of words based of precompiled dictionaries
-  - Management of custom dictionary entries
-  - Next-word suggestions by training language models. Data collected here is stored locally and never leaves
-    the user's device.
-
-- Module C: Extension packs (Implemented with [#162], reworked several times and still not stable)
-  - Ability to load dictionaries (and later potentially other cool
-    features too) only if needed to keep the core APK size small
-  - Currently unclear how exactly this will work, but this is definitely
-    a must-have feature
-  - A full implementation may come only in v0.5.0
-
-- Module D: Glide typing (Implemented with [#544])
-  - Swiping over the characters will automatically convert this to a word
-  - Possibly also add improvements based on the Flow keyboard
-
-- Module E: Theme rework (Implemented with [#162])
-  - Themes are now based on the Asset schema
-  - Dynamic theme creation
-  - Different theme modes (`Always day`, `Always night`, `Follow system`
-    and `Follow time`)
-  - Define a separate theme both for day and night theme
-  - Adapt to app theme if possible
-  - Theme import/export
-
-### [v0.5.0](https://github.com/florisboard/florisboard/milestone/5)
-There's no exact roadmap yet, but these are the most important points:
-- Full layout customization in runtime
-- Extensive rework and customization of the media input (emojis, emoticons, kaomoji)
-- Better Smartbar customization
-- As an extension GIF support
-
-### > v0.5.0
-This is completely open as of now and will gather planned features as time
-passes...
-
-Backlog (currently not assigned to any milestone):
-
-- Floating keyboard
-
-[#91]: https://github.com/florisboard/florisboard/pull/91
-[#162]: https://github.com/florisboard/florisboard/pull/162
-[#329]: https://github.com/florisboard/florisboard/pull/329
-[#544]: https://github.com/florisboard/florisboard/pull/544
+See the [roadmap page](ROADMAP.md) for this.
 
 ## Contributing
 Wanna contribute to FlorisBoard? That's great to hear! There are lots of
@@ -199,21 +143,10 @@ to get more information on this topic.
   [JakeWharton](https://github.com/JakeWharton)
 * [expandable-fab](https://github.com/nambicompany/expandable-fab) by
   [Nambi](https://github.com/nambicompany)
-
-## Usage notes for included binary dictionary files
-All binary dictionaries included within this project in
-(this)[app/src/main/assets/ime/dict] asset folder are built from various
-sources, as stated below.
-
-### Source 1: [wordfreq library by LuminosoInsight](https://github.com/LuminosoInsight/wordfreq):
-`wordfreq` is a repository which provides both a Python library and raw
-data (the wordlists). Only the data has been extracted in order to build
-binary dictionary files from it. `wordfreq`'s data is licensed under the
-Creative Commons Attribution-ShareAlike 4.0 license
-(https://creativecommons.org/licenses/by-sa/4.0/).
-
-For further information on what wordfreq's data depends on, see
-(https://github.com/LuminosoInsight/wordfreq#license).
+* [ICU4C](https://github.com/unicode-org/icu) by
+  [The Unicode Consortium](https://github.com/unicode-org)
+* [Nuspell](https://github.com/nuspell/nuspell) by
+  [Nuspell](https://github.com/nuspell)
 
 ## License
 ```

ROADMAP.md

@@ -0,0 +1,106 @@
+# FlorisBoard's feature roadmap & milestones
+
+This feature roadmap intents to provide transparency to what I want to add
+to FlorisBoard in the foreseeable future. Note that there are no ETAs for any
+version milestones down below, experience says these won't hold anyways.
+
+I try my best to release regularly, though some features take a lot longer
+than others and thus releases can be spaced out a bit on the stable track.
+If you are interested in following the development more closely, make sure to
+follow along the beta track releases! These are generally more unstable but
+you get new stuff faster and can provide early feedback, which helps a lot!
+
+## 0.3.x and 0.4.0
+Releases in this section still follow the old versioning scheme, meaning the
+patch number is a feature upgrade. As this naming convention is more confusing
+than useful, after the v0.4.0 release a new release/development cycle will be
+introduced.
+
+### 0.3.13 (currently in development and soon done)
+- Spell checking (mainly completed and relatively well working, Smartbar integration still missing)
+- Performance improvements in keyboard rendering
+- Audio/haptic feedback rework
+- Lots and lots of bug fixing in all areas, really fix some annoying bugs
+- New layouts added by contributors
+
+### 0.3.14
+- Re-write of the Preference core
+  - Reduce redundancy in key/default value definitions
+  - Avoid having to manually add redundant code for adding a new pref
+  - Goes hand-in-hand with the Settings UI re-write
+- Re-write of the Settings UI with Jetpack Compose
+  - Also re-structure UI into a more list-like panel
+  - Adjust theme colors of Settings a bit to make it more modern
+  - Preview the keyboard at any time from within the Settings
+  - Settings language different than device language
+- Re-write the Setup UI in Jetpack Compose
+  - Simplify screen based on previously discussed ideas and mock-ups
+  - Improve backend setup logic
+- Implement base-UI for extensions and further continue development
+  of existing Flex (FlorisBoard extension) format
+  - Allows for a continuous experience of customizing FlorisBoard in different areas
+  - Planned in the future (not in this version though) what will use Flex:
+    - Themes
+    - Layouts (Characters, symbols, numeric, ...)
+    - Composers for non-Latin script languages
+    - Word suggestion dictionaries
+    - Spell check dictionaries
+    - User dictionaries
+    - Other features that require only data and no logic
+- Maybe full backup of preferences? Not 100% confirmed though and may be pushed back
+
+### 0.3.15
+- Re-adding word suggestions (at least for Latin-based languages at first)
+  - Importing the dictionaries as well as management relies on the Flex extension core and UI in Kotlin
+  - Actually parsing and generating suggestions happens in C++ to avoid another OOM catastrophe like in 0.3.9/10
+  - The actual format of the dictionary and word list source is not decided yet
+- Improvement of the candidate view in Smartbar (for word suggestions)
+- Theme rework part I:
+  - Custom key corner radius
+  - Custom key border color (not shadow!!)
+  - Re-work theme internals so they use Flex format
+  - Community repository on GitHub for theme sharing across users (when Theme Flex format is ready)
+
+### 0.4.0
+- Prepare FlorisBoard repository and app store presence for public beta release
+  on Google Play
+- Rework branding images and texts of FlorisBoard for the app stores
+- Focus on polishing the app and fixing bugs/crashes
+
+With this release the versioning scheme changes: the second number now indicates new features,
+changes in the third "patch" number now indicates bug fixes for the stable track. The development
+cycle for each 0.x release will have -betaXX and -rcXX (release candidate) releases on the beta
+track for interested people to follow along the development.
+
+## 0.5.0
+- Complete rework of the Emoji panel
+  - Recently used / Emoji history
+  - Emoji search
+  - Emoji suggestions when using :emoji_name: syntax
+  - Kaomoji panel implementation (the third tab which currently has "not yet implemented")
+- Full Smartbar customization
+  - Includes internal rework how Smartbar is build and assembled
+  - Allow for more than one Smartbar / Stackable and Collapsible Smartbars
+  - Customizable quick actions, clipboard row
+
+## 0.6.0
+- Full on-board layout editor which allows users to create their own layouts
+  without writing a JSON file
+- Import/Export of custom layout files packed in Flex extensions
+
+## Backlog / Features that MAY be added
+- Theme rework part II
+- Adaptive themes v2
+- Voice-to-text with Mozilla's open-source voice service
+- Text translation
+- Glide typing better word detection
+- Proximity-based key typo detection
+- Floating keyboard
+- Tablet mode / Optimizations for landscape input
+- Stickers/GIFs
+- FlorisBoard landing web page for presentation
+- Implementing additional layouts
+- Support for Tasker/Automate/MacroDroid plugins
+- Support for WearOS/Smartwatches
+- Handwriting
+- ...

app/build.gradle.kts

@@ -1,14 +1,14 @@
-
 plugins {
-    id("com.android.application") version "4.2.1"
-    kotlin("android") version "1.5.0"
-    kotlin("kapt") version "1.5.0"
-    kotlin("plugin.serialization") version "1.5.0"
+    id("com.android.application") version "7.0.1"
+    kotlin("android") version "1.5.20"
+    kotlin("kapt") version "1.5.20"
+    kotlin("plugin.serialization") version "1.5.20"
 }
 
 android {
-    compileSdkVersion(30)
-    buildToolsVersion("30.0.3")
+    compileSdk = 30
+    buildToolsVersion = "30.0.3"
+    ndkVersion = "22.1.7171670"
 
     compileOptions {
         sourceCompatibility = JavaVersion.VERSION_1_8
@@ -17,15 +17,15 @@ android {
 
     kotlinOptions {
         jvmTarget = JavaVersion.VERSION_1_8.toString()
-        freeCompilerArgs = listOf("-Xallow-result-return-type", "-Xopt-in=kotlin.RequiresOptIn")
+        freeCompilerArgs = listOf("-Xallow-result-return-type", "-Xopt-in=kotlin.RequiresOptIn", "-Xopt-in=kotlin.contracts.ExperimentalContracts")
     }
 
     defaultConfig {
         applicationId = "dev.patrickgold.florisboard"
-        minSdkVersion(23)
-        targetSdkVersion(30)
-        versionCode(46)
-        versionName("0.3.13")
+        minSdk = 23
+        targetSdk = 30
+        versionCode = 53
+        versionName = "0.3.13"
 
         testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
 
@@ -41,10 +41,27 @@ android {
 
         externalNativeBuild {
             cmake {
-                cppFlags("-std=c++17", "-fexceptions", "-frtti")
+                cFlags("-fvisibility=hidden", "-DU_STATIC_IMPLEMENTATION=1")
+                cppFlags("-fvisibility=hidden", "-std=c++17", "-fexceptions", "-ffunction-sections", "-fdata-sections", "-DU_DISABLE_RENAMING=1", "-DU_STATIC_IMPLEMENTATION=1")
                 arguments("-DANDROID_STL=c++_static")
             }
         }
+
+        ndk {
+            //abiFilters += listOf("x86", "x86_64", "armeabi-v7a", "arm64-v8a")
+            abiFilters += listOf("armeabi-v7a", "arm64-v8a")
+        }
+
+        sourceSets {
+            maybeCreate("main").apply {
+                assets {
+                    srcDirs("src/main/assets", "src/main/icu4c/prebuilt/assets")
+                }
+                jniLibs {
+                    srcDirs("src/main/icu4c/prebuilt/jniLibs")
+                }
+            }
+        }
     }
 
     buildFeatures {
@@ -73,7 +90,7 @@ android {
         create("beta") // Needed because by default the "beta" BuildType does not exist
         named("beta").configure {
             applicationIdSuffix = ".beta"
-            versionNameSuffix = "-beta03"
+            versionNameSuffix = "-beta10"
             proguardFiles.add(getDefaultProguardFile("proguard-android-optimize.txt"))
 
             resValue("mipmap", "floris_app_icon", "@mipmap/ic_app_icon_beta")
@@ -96,12 +113,11 @@ android {
         }
     }
 
-    lintOptions {
+    lint {
         isAbortOnError = false
     }
 }
 
-
 dependencies {
     implementation("androidx.activity", "activity-ktx", "1.2.1")
     implementation("androidx.appcompat", "appcompat", "1.2.0")
@@ -111,7 +127,7 @@ dependencies {
     implementation("androidx.preference", "preference-ktx", "1.1.1")
     implementation("androidx.constraintlayout", "constraintlayout", "2.0.4")
     implementation("androidx.lifecycle", "lifecycle-service", "2.2.0")
-    implementation("com.google.android", "flexbox", "2.0.1")
+    implementation("com.google.android.flexbox", "flexbox", "3.0.0")
     implementation("com.google.android.material", "material", "1.3.0")
     implementation("org.jetbrains.kotlinx", "kotlinx-coroutines-android", "1.4.2")
     implementation("org.jetbrains.kotlinx", "kotlinx-serialization-json", "1.1.0")
@@ -121,9 +137,11 @@ dependencies {
     implementation("androidx.room", "room-runtime", "2.2.6")
     kapt("androidx.room", "room-compiler","2.2.6")
 
-    testImplementation("junit", "junit", "4.13.1")
+    testImplementation(kotlin("test"))
+    testImplementation("androidx.test", "core", "1.3.0")
     testImplementation("org.mockito", "mockito-inline", "3.7.7")
     testImplementation("org.robolectric", "robolectric", "4.5.1")
+
     androidTestImplementation("androidx.test.ext", "junit", "1.1.2")
     androidTestImplementation("androidx.test.espresso", "espresso-core", "3.3.0")
 }

app/src/main/AndroidManifest.xml

@@ -33,13 +33,27 @@
         <service
             android:name="dev.patrickgold.florisboard.FlorisImeService"
             android:label="@string/floris_app_name"
-            android:permission="android.permission.BIND_INPUT_METHOD">
+            android:permission="android.permission.BIND_INPUT_METHOD"
+            android:directBootAware="true"
+            android:exported="true">
             <intent-filter>
                 <action android:name="android.view.InputMethod"/>
             </intent-filter>
             <meta-data android:name="android.view.im" android:resource="@xml/method"/>
         </service>
 
+        <!-- Spellchecker service -->
+        <service
+            android:name="dev.patrickgold.florisboard.FlorisSpellCheckerService"
+            android:label="@string/floris_app_name"
+            android:permission="android.permission.BIND_TEXT_SERVICE"
+            android:exported="true">
+            <intent-filter>
+                <action android:name="android.service.textservice.SpellCheckerService"/>
+            </intent-filter>
+            <meta-data android:name="android.view.textservice.scs" android:resource="@xml/spellchecker"/>
+        </service>
+
         <!-- Settings Activity -->
         <activity
             android:name="dev.patrickgold.florisboard.settings.SettingsMainActivity"
@@ -56,7 +70,8 @@
             android:label="@string/floris_app_name"
             android:launchMode="singleTask"
             android:roundIcon="@mipmap/floris_app_icon_round"
-            android:targetActivity="dev.patrickgold.florisboard.setup.SetupActivity">
+            android:targetActivity="dev.patrickgold.florisboard.setup.SetupActivity"
+            android:exported="true">
             <intent-filter>
                 <action android:name="android.intent.action.MAIN"/>
                 <category android:name="android.intent.category.LAUNCHER"/>
@@ -84,6 +99,14 @@
             android:label="@string/settings__theme_editor__title"
             android:theme="@style/SettingsTheme"/>
 
+        <!-- Spelling Activity -->
+        <activity
+            android:name="dev.patrickgold.florisboard.settings.spelling.SpellingActivity"
+            android:icon="@mipmap/floris_app_icon"
+            android:label="@string/settings__spelling__title_overview"
+            android:roundIcon="@mipmap/floris_app_icon_round"
+            android:theme="@style/SettingsTheme"/>
+
         <!-- About Activity -->
         <activity
             android:name="dev.patrickgold.florisboard.settings.AboutActivity"

app/src/main/assets/ime/config.json

@@ -2,7 +2,8 @@
   "package": "dev.patrickgold.florisboard",
   "composers": [
     { "$": "appender" },
-    { "$": "hangul-unicode" }
+    { "$": "hangul-unicode" },
+    { "$": "kana-unicode" }
   ],
   "currencySets": [
     {
@@ -682,6 +683,27 @@
       "preferred": {
         "characters": "korean"
       }
+    },
+    {
+      "id": 3000,
+      "languageTag": "lt-LT",
+      "composer": "appender",
+      "currencySet": "euro",
+      "preferred": {
+        "characters": "qwerty"
+      }
+    },
+    {
+      "id": 3100,
+      "languageTag": "ja-JP-jis",
+      "composer": "kana-unicode",
+      "currencySet": "yen",
+      "preferred": {
+        "characters": "jis",
+        "symbols": "cjk",
+        "symbols2": "cjk",
+        "numericRow": "cjk"
+      }
     }
   ]
 }

app/src/main/assets/ime/spelling/config.json

@@ -0,0 +1,69 @@
+{
+  "basePath": "ime/spelling",
+  "importSources": [
+    {
+      "id": "mozilla_firefox",
+      "label": "Mozilla Firefox Add-ons",
+      "url": "https://addons.mozilla.org/firefox/language-tools/",
+      "format": {
+        "$": "archive",
+        "file": {
+          "name": "^.+\\.xpi$",
+          "isRequired": true
+        }
+      }
+    },
+    {
+      "id": "libre_office",
+      "label": "LibreOffice [CURRENTLY UNSUPPORTED]",
+      "url": "https://extensions.libreoffice.org/?Tags%5B%5D=50",
+      "format": {
+        "$": "archive",
+        "file": {
+          "name": "^.+\\.oxt$",
+          "isRequired": true
+        }
+      }
+    },
+    {
+      "id": "open_office",
+      "label": "Apache OpenOffice [CURRENTLY UNSUPPORTED]",
+      "url": "https://extensions.openoffice.org/en/search?f%5B0%5D=field_project_tags%3A157",
+      "format": {
+        "$": "archive",
+        "file": {
+          "name": "^.+\\.oxt$",
+          "isRequired": true
+        }
+      }
+    },
+    {
+      "id": "free_office",
+      "label": "SoftMaker FreeOffice",
+      "url": "https://www.freeoffice.com/en/download/dictionaries",
+      "format": {
+        "$": "archive",
+        "file": {
+          "name": "^.+\\.sox$",
+          "isRequired": true
+        }
+      }
+    },
+    {
+      "id": "gh_wooorm",
+      "label": "GitHub collection by Titus Wormer",
+      "url": "https://github.com/wooorm/dictionaries",
+      "format": {
+        "$": "raw",
+        "affFile": {
+          "name": "^.+\\.aff$",
+          "isRequired": true
+        },
+        "dicFile": {
+          "name": "^.+\\.dic$",
+          "isRequired": true
+        }
+      }
+    }
+  ]
+}

app/src/main/assets/ime/text/characters/extended_popups/ar.json

@@ -4,59 +4,21 @@
   "authors": [ "HeiWiper" ],
   "mapping": {
     "all": {
-      "ض": {
-        "relevant": [
-          { "code": 1633, "label": "١" }
-        ]
-      },
-      "ص": {
-        "relevant": [
-          { "code": 1634, "label": "٢" }
-        ]
-      },
-      "ث": {
-        "relevant": [
-          { "code": 1635, "label": "٣" }
-        ]
-      },
       "ق": {
         "relevant": [
-          { "code": 1704, "label": "ڨ" },
-          { "code": 1636, "label": "٤" }
+          { "code": 1704, "label": "ڨ" }
         ]
       },
       "ف": {
         "relevant": [
           { "code": 1701, "label": "ڥ" },
           { "code": 1700, "label": "ڤ" },
-          { "code": 1698, "label": "ڢ" },
-          { "code": 1637, "label": "٥" }
-        ]
-      },
-      "غ": {
-        "relevant": [
-          { "code": 1638, "label": "٦" }
-        ]
-      },
-      "ع": {
-        "relevant": [
-          { "code": 1639, "label": "٧" }
+          { "code": 1698, "label": "ڢ" }
         ]
       },
       "ه": {
         "relevant": [
-          { "code": 1726, "label": "ھ" },
-          { "code": 1640, "label": "٨" }
-        ]
-      },
-      "خ": {
-        "relevant": [
-          { "code": 1641, "label": "٩" }
-        ]
-      },
-      "ح": {
-        "relevant": [
-          { "code": 1632, "label": "٠" }
+          { "code": 1726, "label": "ھ" }
         ]
       },
       "ج": {
@@ -70,8 +32,8 @@
         ]
       },
       "ي": {
+        "main": { "code": 1574, "label": "ئ" },
         "relevant": [
-          { "code": 1574, "label": "ئ" },
           { "code": 1609, "label": "ى" }
         ]
       },
@@ -89,10 +51,10 @@
         ]
       },
       "ا": {
+        "main": { "code": 1571, "label": "أ" },
         "relevant": [
           { "code": 1570, "label": "آ" },
           { "code": 1569, "label": "ء" },
-          { "code": 1571, "label": "أ" },
           { "code": 1573, "label": "إ" },
           { "code": 1649, "label": "ٱ" }
         ]
@@ -104,9 +66,7 @@
         ]
       },
       "ى": {
-        "relevant": [
-          { "code": 1574, "label": "ئ" }
-        ]
+        "main": { "code": 1574, "label": "ئ" }
       },
       "ز": {
         "relevant": [

app/src/main/assets/ime/text/characters/extended_popups/ckb.json

@@ -4,28 +4,22 @@
   "authors": [ "GoRaN" ],
   "mapping": {
     "all": {
-
       "ﻪ": {
         "relevant": [
-          { "code": 1577, "label": "ة" },
           { "code": 1729, "label": "ـہ" }
         ]
       },
       "ر": {
         "relevant": [
-          { "code": 1685, "label": "ڕ" },
           { "code": 1682, "label": "ڒ" }
         ]
       },
-
-
       "ی": {
         "relevant": [
           { "code": 1746, "label": "ے" },
-          { "code": 1610, "label": "ي" },
-          { "code": 1744, "label": "ې" },
           { "code": 1741, "label": "ۍ" },
-          { "code": 1742, "label": "ێ" },
+          { "code": 1744, "label": "ې" },
+          { "code": 1610, "label": "ي" },
           { "code": 1597, "label": "ؽ" }
         ]
       },
@@ -34,10 +28,15 @@
       "ﺋ": {
         "relevant": [
           { "code": 65163, "label": "ﺋ" },
-          { "code": 1569, "label": "ء" },
           { "code": 65139, "label": "ﹳ" }
         ]
       },
+      "ح": {
+        "relevant": [
+          { "code": 65010, "label": "ﷲ" },
+          { "code": 65019, "label": "ﷻ" }
+        ]
+      },
 
       "ع": {
         "relevant": [
@@ -56,12 +55,9 @@
         ]
       },
 
-
-
       "ف": {
         "relevant": [
           { "code": 1701, "label": "ڥ" },
-          { "code": 1700, "label": "ڤ" },
           { "code": 1698, "label": "ڢ" },
           { "code": 1697, "label": "ڡ" }
         ]
@@ -70,7 +66,6 @@
       "د": {
         "relevant": [
           { "code": 1676, "label": "ڌ" },
-          { "code": 1584, "label": "ذ" },
           { "code": 64390, "label": "ﮆ" },
           { "code": 1774, "label": "ۮ" }
         ]
@@ -93,9 +88,7 @@
       },
       "ب": {
         "relevant": [
-          { "code": 65010, "label": "ﷲ" },
-          { "code": 65021, "label": "﷽" },
-          { "code": 65019, "label": "ﷻ" }
+          { "code": 65021, "label": "﷽" }
         ]
       },
       "م": {
@@ -108,7 +101,6 @@
         "relevant": [
           { "code": 1718, "label": "ڶ" },
           { "code": 1719, "label": "ڷ" },
-          { "code": 1717, "label": "ڵ" },
           { "code": 1720, "label": "ڸ" }
         ]
       },

app/src/main/assets/ime/text/characters/extended_popups/ja-JP-jis.json

@@ -0,0 +1,292 @@
+{
+  "type": "characters/extended_popups",
+  "name": "ja-JP-jis",
+  "authors": [ "waelwindows" ],
+  "mapping": {
+    "all": {
+      "あ": {
+        "main": { "code": 12353, "label": "ぁ" }
+      },
+      "ア": {
+        "main": { "code": 12449, "label": "ァ" }
+      },
+      "ｱ": {
+        "main": { "code": 65383, "label": "ｧ" }
+      },
+      "い": {
+        "main": { "code": 12355, "label": "ぃ" },
+        "relevant": [
+          { "code":  12432, "label": "ゐ" },
+          { "code": 110928, "label": "𛅐" }
+        ]
+      },
+      "イ": {
+        "main": { "code": 12451, "label": "ィ" },
+        "relevant": [
+          { "code":  12528, "label": "ヰ" },
+          { "code": 110948, "label": "𛅤" }
+        ]
+      },
+      "ｲ": {
+        "main": { "code": 65384, "label": "ｨ" }
+      },
+      "う": {
+        "main": { "code": 12357, "label": "ぅ" }
+      },
+      "ウ": {
+        "main": { "code": 12453, "label": "ゥ" }
+      },
+      "ｳ": {
+        "main": { "code": 65385, "label": "ｩ" }
+      },
+      "え": {
+        "main": { "code": 12359, "label": "ぇ" },
+        "relevant": [
+          { "code":  12433, "label": "ゑ" },
+          { "code": 110929, "label": "𛅑" }
+        ]
+      },
+      "エ": {
+        "main": { "code": 12455, "label": "ェ" },
+        "relevant": [
+          { "code":  12529, "label": "ヱ" },
+          { "code": 110949, "label": "𛅥" }
+        ]
+      },
+      "ｴ": {
+        "main": { "code": 65386, "label": "ｪ" }
+      },
+      "お": {
+        "main": { "code": 12361, "label": "ぉ" }
+      },
+      "オ": {
+        "main": { "code": 12457, "label": "ォ" }
+      },
+      "ｵ": {
+        "main": { "code": 65387, "label": "ｫ" }
+      },
+      "や": {
+        "main": { "code": 12419, "label": "ゃ" }
+      },
+      "ヤ": {
+        "main": { "code": 12515, "label": "ャ" }
+      },
+      "ﾔ": {
+        "main": { "code": 65388, "label": "ｬ" }
+      },
+      "ゆ": {
+        "main": { "code": 12421, "label": "ゅ" }
+      },
+      "ユ": {
+        "main": { "code": 12517, "label": "ュ" }
+      },
+      "ﾕ": {
+        "main": { "code": 65389, "label": "ｭ" }
+      },
+      "よ": {
+        "main": { "code": 12423, "label": "ょ" }
+      },
+      "ヨ": {
+        "main": { "code": 12519, "label": "ョ" }
+      },
+      "ﾖ": {
+        "main": { "code": 65390, "label": "ｮ" }
+      },
+      "わ": {
+        "main": { "code": 12434, "label": "を" },
+        "relevant": [
+          { "code":  12430, "label": "ゎ" },
+          { "code": 110930, "label": "𛅒" }
+        ]
+      },
+      "ワ": {
+        "main": { "code": 12530, "label": "ヲ" },
+        "relevant": [
+          { "code":  12526, "label": "ヮ" },
+          { "code": 110950, "label": "𛅦" }
+        ]
+      },
+      "ﾜ": {
+        "main": { "code": 65382, "label": "ｦ" }
+      },
+      "つ": {
+        "main": { "code": 12387, "label": "っ" }
+      },
+      "ツ": {
+        "main": { "code": 12483, "label": "ッ" }
+      },
+      "ﾂ": {
+        "main": { "code": 65391, "label": "ｯ" }
+      },
+      "ト": {
+        "relevant": [
+          { "code": 12787, "label": "ㇳ" }
+        ]
+      },
+      "シ": {
+        "relevant": [
+          { "code": 12785, "label": "ㇱ" }
+        ]
+      },
+      "ス": {
+        "main": { "code": 12786, "label": "ㇲ" }
+      },
+      "か": {
+        "main": { "code": 12437, "label": "ゕ" }
+      },
+      "カ": {
+        "main": { "code": 12533, "label": "ヵ" }
+      },
+      "ク": {
+        "main": { "code": 12784, "label": "ㇰ" }
+      },
+      "け": {
+        "main": { "code": 12438, "label": "ゖ" }
+      },
+      "ケ": {
+        "main": { "code": 12534, "label": "ヶ" }
+      },
+      "ヌ": {
+        "relevant": [
+          { "code": 12788, "label": "ㇴ" }
+        ]
+      },
+      "ハ": {
+        "relevant": [
+          { "code": 12789, "label": "ㇵ" }
+        ]
+      },
+      "ヒ": {
+        "relevant": [
+          { "code": 12790, "label": "ㇶ" }
+        ]
+      },
+      "フ": {
+        "relevant": [
+          { "code": 12791, "label": "ㇷ" }
+        ]
+      },
+      "ヘ": {
+        "relevant": [
+          { "code": 12792, "label": "ㇸ" }
+        ]
+      },
+      "ホ": {
+        "relevant": [
+          { "code": 12793, "label": "ㇹ" }
+        ]
+      },
+      "ム": {
+        "relevant": [
+          { "code": 12794, "label": "ㇺ" }
+        ]
+      },
+      "ラ": {
+        "relevant": [
+          { "code": 12795, "label": "ㇻ" }
+        ]
+      },
+      "リ": {
+        "relevant": [
+          { "code": 12796, "label": "ㇼ" }
+        ]
+      },
+      "ル": {
+        "relevant": [
+          { "code": 12797, "label": "ㇽ" }
+        ]
+      },
+      "レ": {
+        "relevant": [
+          { "code": 12798, "label": "ㇾ" }
+        ]
+      },
+      "ロ": {
+        "relevant": [
+          { "code": 12799, "label": "ㇿ" }
+        ]
+      },
+      "゛": {
+        "main": { "code": 12443, "label": "゛" }
+      },
+      "゜": {
+        "main": { "code": 12444, "label": "゜" }
+      },
+      "~kana": {
+        "relevant": [
+          { "code": -9711, "label": "あ" },
+          { "code": -9712, "label": "ア" },
+          { "code": -9713, "label": "ｱ" }
+        ]
+      },
+      "~right": {
+        "main": { "$": "char_width_selector",
+          "full": { "code": 12289, "label": "、" },
+          "half": { "code": 65380, "label": "､" }
+        },
+        "relevant": [
+          { "$": "char_width_selector",
+            "full": { "code": 65286, "label": "＆" },
+            "half": { "code":    38, "label": "&" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65285, "label": "％" },
+            "half": { "code":    37, "label": "%" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65291, "label": "＋" },
+            "half": { "code":    43, "label": "+" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65340, "label": "＼" },
+            "half": { "code":    92, "label": "\\" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65293, "label": "－" },
+            "half": { "code":    45, "label": "-" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65306, "label": "：" },
+            "half": { "code":    58, "label": ":" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65287, "label": "＇" },
+            "half": { "code":    39, "label": "'" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65312, "label": "＠" },
+            "half": { "code":    64, "label": "@" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65307, "label": "；" },
+            "half": { "code":    59, "label": ";" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65295, "label": "／" },
+            "half": { "code":    47, "label": "/" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65288, "label": "（" },
+            "half": { "code":    40, "label": "(" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65289, "label": "）" },
+            "half": { "code":    41, "label": ")" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65283, "label": "＃" },
+            "half": { "code":    35, "label": "#" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65281, "label": "！" },
+            "half": { "code":    33, "label": "!" }
+          },
+          { "$": "char_width_selector",
+            "full": { "code": 65311, "label": "？" },
+            "half": { "code":    63, "label": "?" }
+          }
+        ]
+      }
+    }
+  }
+}

app/src/main/assets/ime/text/characters/extended_popups/lt.json

@@ -0,0 +1,167 @@
+{
+  "type": "characters/extended_popups",
+  "name": "lv",
+  "authors": [ "patrickgold" ],
+  "mapping": {
+    "all": {
+      "a": {
+        "main": { "$": "auto_text_key", "code":  261, "label": "ą" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  226, "label": "â" },
+          { "$": "auto_text_key", "code":  227, "label": "ã" },
+          { "$": "auto_text_key", "code":  229, "label": "å" },
+          { "$": "auto_text_key", "code":  230, "label": "æ" },
+          { "$": "auto_text_key", "code":  228, "label": "ä" },
+          { "$": "auto_text_key", "code":  257, "label": "ā" },
+          { "$": "auto_text_key", "code":  224, "label": "à" },
+          { "$": "auto_text_key", "code":  225, "label": "á" }
+        ]
+      },
+      "c": {
+        "main": { "$": "auto_text_key", "code":  269, "label": "č" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  263, "label": "ć" },
+          { "$": "auto_text_key", "code":  231, "label": "ç" }
+        ]
+      },
+      "d": {
+        "main": { "$": "auto_text_key", "code":  271, "label": "ď" }
+      },
+      "e": {
+        "main": { "$": "auto_text_key", "code":  279, "label": "ė" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  235, "label": "ë" },
+          { "$": "auto_text_key", "code":  233, "label": "é" },
+          { "$": "auto_text_key", "code":  234, "label": "ê" },
+          { "$": "auto_text_key", "code":  283, "label": "ě" },
+          { "$": "auto_text_key", "code":  275, "label": "ē" },
+          { "$": "auto_text_key", "code":  281, "label": "ę" },
+          { "$": "auto_text_key", "code":  232, "label": "è" }
+        ]
+      },
+      "g": {
+        "main": { "$": "auto_text_key", "code":  291, "label": "ģ" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  287, "label": "ğ" }
+        ]
+      },
+      "i": {
+        "main": { "$": "auto_text_key", "code":  303, "label": "į" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  238, "label": "î" },
+          { "$": "auto_text_key", "code":  239, "label": "ï" },
+          { "$": "auto_text_key", "code":  236, "label": "ì" },
+          { "$": "auto_text_key", "code":  299, "label": "ī" },
+          { "$": "auto_text_key", "code":  237, "label": "í" }
+        ]
+      },
+      "k": {
+        "main": { "$": "auto_text_key", "code":  311, "label": "ķ" }
+      },
+      "l": {
+        "main": { "$": "auto_text_key", "code":  316, "label": "ļ" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  318, "label": "ľ" },
+          { "$": "auto_text_key", "code":  314, "label": "ĺ" },
+          { "$": "auto_text_key", "code":  322, "label": "ł" }
+        ]
+      },
+      "n": {
+        "main": { "$": "auto_text_key", "code":  326, "label": "ņ" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  324, "label": "ń" },
+          { "$": "auto_text_key", "code":  241, "label": "ñ" }
+        ]
+      },
+      "o": {
+        "main": { "$": "auto_text_key", "code":  246, "label": "ö" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  248, "label": "ø" },
+          { "$": "auto_text_key", "code":  337, "label": "ő" },
+          { "$": "auto_text_key", "code":  244, "label": "ô" },
+          { "$": "auto_text_key", "code":  339, "label": "œ" },
+          { "$": "auto_text_key", "code":  243, "label": "ó" },
+          { "$": "auto_text_key", "code":  242, "label": "ò" },
+          { "$": "auto_text_key", "code":  245, "label": "õ" }
+        ]
+      },
+      "r": {
+        "main": { "$": "auto_text_key", "code":  343, "label": "ŗ" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  341, "label": "ŕ" },
+          { "$": "auto_text_key", "code":  345, "label": "ř" }
+        ]
+      },
+      "s": {
+        "main": { "$": "auto_text_key", "code":  353, "label": "š" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  347, "label": "ś" },
+          { "$": "auto_text_key", "code":  223, "label": "ß" },
+          { "$": "auto_text_key", "code":  351, "label": "ş" }
+        ]
+      },
+      "t": {
+        "main": { "$": "auto_text_key", "code":  355, "label": "ţ" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  357, "label": "ť" }
+        ]
+      },
+      "u": {
+        "main": { "$": "auto_text_key", "code":  363, "label": "ū" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  367, "label": "ů" },
+          { "$": "auto_text_key", "code":  250, "label": "ú" },
+          { "$": "auto_text_key", "code":  251, "label": "û" },
+          { "$": "auto_text_key", "code":  369, "label": "ű" },
+          { "$": "auto_text_key", "code":  249, "label": "ù" },
+          { "$": "auto_text_key", "code":  252, "label": "ü" },
+          { "$": "auto_text_key", "code":  371, "label": "ų" }
+        ]
+      },
+      "y": {
+        "main": { "$": "auto_text_key", "code":  253, "label": "ý" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  255, "label": "ÿ" }
+        ]
+      },
+      "z": {
+        "main": { "$": "auto_text_key", "code":  382, "label": "ž" },
+        "relevant": [
+          { "$": "auto_text_key", "code":  378, "label": "ź" },
+          { "$": "auto_text_key", "code":  380, "label": "ż" }
+        ]
+      },
+      "~right": {
+        "main": { "code":   44, "label": "," },
+        "relevant": [
+          { "code":   38, "label": "&" },
+          { "code":   37, "label": "%" },
+          { "code":   43, "label": "+" },
+          { "code":   34, "label": "\"" },
+          { "code":   45, "label": "-" },
+          { "code":   58, "label": ":" },
+          { "code":   39, "label": "'" },
+          { "code":   64, "label": "@" },
+          { "code":   59, "label": ";" },
+          { "code":   47, "label": "/" },
+          { "code":   40, "label": "(" },
+          { "code":   41, "label": ")" },
+          { "code":   35, "label": "#" },
+          { "code":   33, "label": "!" },
+          { "code":   63, "label": "?" }
+        ]
+      }
+    },
+    "uri": {
+      "~right": {
+        "main": { "code": -255, "label": ".lv" },
+        "relevant": [
+          { "code": -255, "label": ".gov" },
+          { "code": -255, "label": ".edu" },
+          { "code": -255, "label": ".com" },
+          { "code": -255, "label": ".net" }
+        ]
+      }
+    }
+  }
+}

app/src/main/assets/ime/text/characters/extended_popups/tr.json

@@ -1,99 +1,35 @@
 {
   "type": "characters/extended_popups",
   "name": "tr",
-  "authors": [ "kisekinopureya", "patrickgold" ],
+  "authors": [ "kisekinopureya", "patrickgold", "dvrnynr" ],
   "mapping": {
     "all": {
-      "a": {
-        "relevant": [
-          { "$": "auto_text_key", "code":  226, "label": "â" },
-          { "$": "auto_text_key", "code":  228, "label": "ä" },
-          { "$": "auto_text_key", "code":  225, "label": "á" }
-        ]
-      },
       "c": {
-        "main": { "$": "auto_text_key", "code":  231, "label": "ç" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  269, "label": "č" },
-          { "$": "auto_text_key", "code":  263, "label": "ć" }
-        ]
-      },
-      "e": {
-        "relevant": [
-          { "$": "auto_text_key", "code":  233, "label": "é" },
-          { "$": "auto_text_key", "code":  601, "label": "ə" },
-          { "$": "auto_text_key", "code":  234, "label": "ê" }
-        ]
+        "main": { "$": "auto_text_key", "code":  231, "label": "ç" }
       },
       "g": {
         "main": { "$": "auto_text_key", "code":  287, "label": "ğ" }
       },
       "i": {
-        "main": { "$": "auto_text_key", "code":  305, "label": "ı" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  303, "label": "į" },
-          { "$": "auto_text_key", "code":  236, "label": "ì" },
-          { "$": "auto_text_key", "code":  237, "label": "í" },
-          { "$": "auto_text_key", "code":  299, "label": "ī" },
-          { "$": "auto_text_key", "code":  238, "label": "î" },
-          { "$": "auto_text_key", "code":  239, "label": "ï" }
-        ]
+        "main": { "$": "case_selector",
+          "lower": { "code":  305, "label": "ı" },
+          "upper": { "code":   73, "label": "I" }
+        }
       },
       "ı": {
-        "main": { "$": "auto_text_key", "code":  105, "label": "i" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  303, "label": "į" },
-          { "$": "auto_text_key", "code":  236, "label": "ì" },
-          { "$": "auto_text_key", "code":  237, "label": "í" },
-          { "$": "auto_text_key", "code":  299, "label": "ī" },
-          { "$": "auto_text_key", "code":  238, "label": "î" },
-          { "$": "auto_text_key", "code":  239, "label": "ï" }
-        ]
-      },
-      "n": {
-        "relevant": [
-          { "$": "auto_text_key", "code":  328, "label": "ň" },
-          { "$": "auto_text_key", "code":  241, "label": "ñ" }
-        ]
+        "main": { "$": "case_selector",
+          "lower": { "code":  105, "label": "i" },
+          "upper": { "code":  304, "label": "İ" }
+        }
       },
       "o": {
-        "main": { "$": "auto_text_key", "code":  246, "label": "ö" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  333, "label": "ō" },
-          { "$": "auto_text_key", "code":  248, "label": "ø" },
-          { "$": "auto_text_key", "code":  243, "label": "ó" },
-          { "$": "auto_text_key", "code":  245, "label": "õ" },
-          { "$": "auto_text_key", "code":  242, "label": "ò" },
-          { "$": "auto_text_key", "code":  339, "label": "œ" },
-          { "$": "auto_text_key", "code":  244, "label": "ô" }
-        ]
+        "main": { "$": "auto_text_key", "code":  246, "label": "ö" }
       },
       "s": {
-        "main": { "$": "auto_text_key", "code":  351, "label": "ş" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  347, "label": "ś" },
-          { "$": "auto_text_key", "code":  223, "label": "ß" },
-          { "$": "auto_text_key", "code":  353, "label": "š" }
-        ]
+        "main": { "$": "auto_text_key", "code":  351, "label": "ş" }
       },
       "u": {
-        "main": { "$": "auto_text_key", "code":  252, "label": "ü" },
-        "relevant": [
-          { "$": "auto_text_key", "code":  363, "label": "ū" },
-          { "$": "auto_text_key", "code":  249, "label": "ù" },
-          { "$": "auto_text_key", "code":  250, "label": "ú" },
-          { "$": "auto_text_key", "code":  251, "label": "û" }
-        ]
-      },
-      "y": {
-        "relevant": [
-          { "$": "auto_text_key", "code":  253, "label": "ý" }
-        ]
-      },
-      "z": {
-        "relevant": [
-          { "$": "auto_text_key", "code":  382, "label": "ž" }
-        ]
+        "main": { "$": "auto_text_key", "code":  252, "label": "ü" }
       },
       "~right": {
         "main": { "code":   44, "label": "," },
@@ -120,10 +56,10 @@
       "~right": {
         "main": { "code": -255, "label": ".com" },
         "relevant": [
-          { "code": -255, "label": ".gov" },
+          { "code": -255, "label": ".gov.tr" },
           { "code": -255, "label": ".org" },
-          { "code": -255, "label": ".edu" },
-          { "code": -255, "label": ".tr" },
+          { "code": -255, "label": ".edu.tr" },
+          { "code": -255, "label": ".com.tr" },
           { "code": -255, "label": ".net" }
         ]
       }

app/src/main/assets/ime/text/characters/halmak.json

@@ -0,0 +1,77 @@
+{
+  "type": "characters",
+  "name": "halmak",
+  "label": "Halmak",
+  "authors": [ "dessalines" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "$": "auto_text_key", "code":  119, "label": "w" },
+      { "$": "auto_text_key", "code":  108, "label": "l" },
+      { "$": "auto_text_key", "code":  114, "label": "r" },
+      { "$": "auto_text_key", "code":   98, "label": "b" },
+      { "$": "auto_text_key", "code":  122, "label": "z" },
+      { "$": "case_selector",
+        "lower": { "code":   59, "label": ";", "popup": {
+          "relevant": [
+            { "code":   58, "label": ":" }
+          ]
+        } },
+        "upper": { "code":   58, "label": ":", "popup": {
+          "relevant": [
+            { "code":   59, "label": ";" }
+          ]
+        } }
+      },
+      { "$": "auto_text_key", "code":  113, "label": "q" },
+      { "$": "auto_text_key", "code":  117, "label": "u" },
+      { "$": "auto_text_key", "code":  100, "label": "d" },
+      { "$": "auto_text_key", "code":  106, "label": "j" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  115, "label": "s" },
+      { "$": "auto_text_key", "code":  104, "label": "h" },
+      { "$": "auto_text_key", "code":  110, "label": "n" },
+      { "$": "auto_text_key", "code":  116, "label": "t" },
+      { "$": "case_selector",
+        "lower": { "code":   44, "label": ",", "popup": {
+          "relevant": [
+            { "code":   40, "label": "(" }
+          ]
+        } },
+        "upper": { "code":   40, "label": "(", "popup": {
+          "relevant": [
+            { "code":   44, "label": "," }
+          ]
+        } }
+      },
+      { "$": "case_selector",
+        "lower": { "code":   46, "label": ".", "popup": {
+          "relevant": [
+            { "code":   41, "label": ")" }
+          ]
+        } },
+        "upper": { "code":   41, "label": ")", "popup": {
+          "relevant": [
+            { "code":   46, "label": "." }
+          ]
+        } }
+      },
+      { "$": "auto_text_key", "code":   97, "label": "a" },
+      { "$": "auto_text_key", "code":  101, "label": "e" },
+      { "$": "auto_text_key", "code":  111, "label": "o" },
+      { "$": "auto_text_key", "code":  105, "label": "i" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  102, "label": "f" },
+      { "$": "auto_text_key", "code":  109, "label": "m" },
+      { "$": "auto_text_key", "code":  118, "label": "v" },
+      { "$": "auto_text_key", "code":   99, "label": "c" },
+      { "$": "auto_text_key", "code":  103, "label": "g" },
+      { "$": "auto_text_key", "code":  112, "label": "p" },
+      { "$": "auto_text_key", "code":  120, "label": "x" },
+      { "$": "auto_text_key", "code":  107, "label": "k" },
+      { "$": "auto_text_key", "code":  121, "label": "y" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/characters/jis.json

@@ -0,0 +1,354 @@
+{
+  "type": "characters",
+  "name": "jis",
+  "label": "JIS",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "modifier": "jis",
+  "arrangement": [
+    [
+      { "$": "kana_selector",
+        "hira": { "code": 12396, "label": "ぬ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12492, "label": "ヌ" },
+          "half": { "code": 65415, "label": "ﾇ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12405, "label": "ふ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12501, "label": "フ" },
+          "half": { "code": 65420, "label": "ﾌ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12354, "label": "あ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12450, "label": "ア" },
+          "half": { "code": 65393, "label": "ｱ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12358, "label": "う" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12454, "label": "ウ" },
+          "half": { "code": 65395, "label": "ｳ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12360, "label": "え" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12456, "label": "エ" },
+          "half": { "code": 65396, "label": "ｴ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12362, "label": "お" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12458, "label": "オ" },
+          "half": { "code": 65397, "label": "ｵ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12420, "label": "や" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12516, "label": "ヤ" },
+          "half": { "code": 65428, "label": "ﾔ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12422, "label": "ゆ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12518, "label": "ユ" },
+          "half": { "code": 65429, "label": "ﾕ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12424, "label": "よ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12520, "label": "ヨ" },
+          "half": { "code": 65430, "label": "ﾖ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12431, "label": "わ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12527, "label": "ワ" },
+          "half": { "code": 65436, "label": "ﾜ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12411, "label": "ほ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12507, "label": "ホ" },
+          "half": { "code": 65422, "label": "ﾎ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12408, "label": "へ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12504, "label": "ヘ" },
+          "half": { "code": 65421, "label": "ﾍ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12540, "label": "ー" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12540, "label": "ー" },
+          "half": { "code": 65392, "label": "ｰ" }
+        }
+      }
+    ],
+    [
+      { "$": "kana_selector",
+        "hira": { "code": 12383, "label": "た" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12479, "label": "タ" },
+          "half": { "code": 65408, "label": "ﾀ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12390, "label": "て" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12486, "label": "テ" },
+          "half": { "code": 65411, "label": "ﾃ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12356, "label": "い" },
+        "kata": { "$": "char_width_selector",
+             "full": { "code": 12452, "label": "イ" },
+             "half": { "code": 65394, "label": "ｲ" }
+         }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12377, "label": "す" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12473, "label": "ス" },
+          "half": { "code": 65405, "label": "ｽ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12363, "label": "か" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12459, "label": "カ" },
+          "half": { "code": 65398, "label": "ｶ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12435, "label": "ん" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12531, "label": "ン" },
+          "half": { "code": 65437, "label": "ﾝ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12394, "label": "な" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12490, "label": "ナ" },
+          "half": { "code": 65413, "label": "ﾅ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12395, "label": "に" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12491, "label": "ニ" },
+          "half": { "code": 65414, "label": "ﾆ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12425, "label": "ら" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12521, "label": "ラ" },
+          "half": { "code": 65431, "label": "ﾗ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12379, "label": "せ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12475, "label": "セ" },
+          "half": { "code": 65406, "label": "ｾ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12441, "label": "゛" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12441, "label": "゛" },
+          "half": { "code": 65438, "label": "ﾞ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12442, "label": "゜" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12442, "label": "゜" },
+          "half": { "code": 65439, "label": "ﾟ" }
+        }
+      }
+    ],
+    [
+      { "$": "kana_selector",
+        "hira": { "code": 12385, "label": "ち" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12481, "label": "チ" },
+          "half": { "code": 65409, "label": "ﾁ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12392, "label": "と" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12488, "label": "ト" },
+          "half": { "code": 65412, "label": "ﾄ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12375, "label": "し" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12471, "label": "シ" },
+          "half": { "code": 65410, "label": "ｼ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12399, "label": "は" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12495, "label": "ハ" },
+          "half": { "code": 65418, "label": "ﾊ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12365, "label": "き" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12461, "label": "キ" },
+          "half": { "code": 65399, "label": "ｷ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12367, "label": "く" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12463, "label": "ク" },
+          "half": { "code": 65400, "label": "ｸ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12414, "label": "ま" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12510, "label": "マ" },
+          "half": { "code": 65423, "label": "ﾏ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12398, "label": "の" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12494, "label": "ノ" },
+          "half": { "code": 65417, "label": "ﾉ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12426, "label": "り" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12522, "label": "リ" },
+          "half": { "code": 65432, "label": "ﾘ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12428, "label": "れ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12524, "label": "レ" },
+          "half": { "code": 65434, "label": "ﾚ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12369, "label": "け" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12465, "label": "ケ" },
+          "half": { "code": 65401, "label": "ｹ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12416, "label": "む" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12512, "label": "ム" },
+          "half": { "code": 65425, "label": "ﾑ" }
+        }
+      }
+    ],
+    [
+      { "$": "kana_selector",
+        "hira": { "code": 12388, "label": "つ" },
+        "kata": { "$": "char_width_selector",
+             "full": { "code": 12484, "label": "ツ" },
+             "half": { "code": 65410, "label": "ﾂ" }
+         }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12373, "label": "さ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12469, "label": "サ" },
+          "half": { "code": 65403, "label": "ｻ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12381, "label": "そ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12477, "label": "ソ" },
+          "half": { "code": 65407, "label": "ｿ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12402, "label": "ひ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12498, "label": "ヒ" },
+          "half": { "code": 65419, "label": "ﾋ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12371, "label": "こ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12467, "label": "コ" },
+          "half": { "code": 65402, "label": "ｺ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12415, "label": "み" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12511, "label": "ミ" },
+          "half": { "code": 65424, "label": "ﾐ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12418, "label": "も" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12514, "label": "モ" },
+          "half": { "code": 65427, "label": "ﾓ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12397, "label": "ね" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12493, "label": "ネ" },
+          "half": { "code": 65416, "label": "ﾈ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12427, "label": "る" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12523, "label": "ル" },
+          "half": { "code": 65433, "label": "ﾙ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12417, "label": "め" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12513, "label": "メ" },
+          "half": { "code": 65426, "label": "ﾒ" }
+        }
+      },
+      { "$": "kana_selector",
+        "hira": { "code": 12429, "label": "ろ" },
+        "kata": { "$": "char_width_selector",
+          "full": { "code": 12525, "label": "ロ" },
+          "half": { "code": 65435, "label": "ﾛ" }
+        }
+      }
+    ]
+  ]
+}

app/src/main/assets/ime/text/characters/kurdish.json

@@ -1,7 +1,7 @@
 {
   "type": "characters",
   "name": "kurdish",
-  "label": "کوردی",
+  "label": "کوردی (قوەرتی نوێ)",
   "authors": [ "GoRaN" ],
   "direction": "rtl",
   "modifier": "kurdish",
@@ -13,34 +13,46 @@
       { "code":  1608, "label": "و", "popup": {
         "main": { "code": -255, "label": "وو" }
       } },
-      { "code":  1749, "label": "ﻪ" },
-      { "code":  1585, "label": "ر" },
-
+      { "code":  1749, "label": "ﻪ", "popup": {
+        "main": { "code": 1577, "label": "ة" }
+      } },
+      { "code":  1585, "label": "ر", "popup": {
+        "main": { "code": 1685, "label": "ڕ" }
+      } },
       { "code":  1578, "label": "ت", "popup": {
         "main": { "code": 1591, "label": "ط" }
       } },
-      { "code":  1740, "label": "ی" },
-
-      { "code":  1574, "label": "ﺋ"},
-
+      { "code":  1740, "label": "ی", "popup": {
+        "main": { "code": 1742, "label": "ێ" }
+      } },
+      { "code":  1574, "label": "ﺋ", "popup": {
+        "main": { "code": 1569, "label": "ء" }
+      } },
       { "code":  1593, "label": "ع", "popup": {
         "main": { "code": 1594, "label": "غ" }
       } },
       { "code":  1734, "label": "ۆ" },
-
       { "code":  1662, "label": "پ", "popup": {
         "main": { "code": 1579, "label": "ث" }
       } }
     ],
     [
-      { "code":  1575, "label": "ا" },
+      {"code": 1575, "label": "ا"},
       { "code":  1587, "label": "س" },
       { "code":  1588, "label": "ش" },
-      { "code":  1583, "label": "د" },
-      { "code":  1601, "label": "ف" },
+      { "code":  1583, "label": "د", "popup": {
+        "main": {"code": 1584, "label": "ذ" }
+      } },
+      { "code":  1601, "label": "ف" , "popup": {
+        "main": {"code": 1700, "label": "ڤ" }
+      } },
       { "code":  1607, "label": "ھ" },
-      { "code":  1688, "label": "ژ" },
-      { "code":  1604, "label": "ل" },
+      { "code":  1688, "label": "ژ", "popup": {
+        "main": { "code": 1600, "label": "▬" }
+      } },
+      { "code":  1604, "label": "ل", "popup": {
+        "main": { "code": 1717, "label": "ڵ" }
+      } },
       { "code":  1705, "label": "ک" },
       { "code":  1711, "label": "گ" }
     ],

app/src/main/assets/ime/text/characters/kurdish_kurmanci.json

@@ -13,7 +13,7 @@
       { "$": "auto_text_key", "code":  116, "label": "t" },
       { "$": "auto_text_key", "code":  121, "label": "y" },
       { "$": "auto_text_key", "code":  117, "label": "u" },
-      { "$": "auto_text_key", "code":  305, "label": "ı" },
+      { "$": "auto_text_key", "code":  105, "label": "i" },
       { "$": "auto_text_key", "code":  111, "label": "o" },
       { "$": "auto_text_key", "code":  112, "label": "p" },
       { "$": "auto_text_key", "code":  251, "label": "û" }

app/src/main/assets/ime/text/characters/kurdish_standard.json

@@ -1,7 +1,7 @@
 {
   "type": "characters",
   "name": "kurdish_standard",
-  "label": "کوردی - ستاندارد",
+  "label": 	"کوردی (ق‌ڤ‌ف‌غ)",
   "authors": [ "GoRaN" ],
   "direction": "rtl",
   "modifier": "kurdish",
@@ -10,16 +10,14 @@
       { "code":  1602, "label": "ق", "popup": {
         "main": { "code": 1647, "label": "ٯ" }
       } },
-      { "code":  1700, "label": "ڤ", "popup": {
-        "main": { "code": 1701, "label": "ڥ" }
-      } },
-      { "code":  1601, "label": "ف", "popup": {
-        "main": { "code": 1698, "label": "ڢ" }
-      } },
+      { "code":  1700, "label": "ڤ" },
+      { "code":  1601, "label": "ف" },
       { "code":  1594, "label": "غ" },
       { "code":  1593, "label": "ع"},
       { "code":  1607, "label": "ھ" },
-      { "code":  1749, "label": "ﻪ" },
+      { "code":  1749, "label": "ﻪ", "popup": {
+        "main": { "code": 1577, "label": "ة" }
+      } },
 
       { "code":  1578, "label": "ت", "popup": {
         "main": { "code": 1591, "label": "ط" }
@@ -46,7 +44,9 @@
       } },
       { "code":  1585, "label": "ر" },
       { "code":  1685, "label": "ڕ" },
-      { "code":  1583, "label": "د" },
+      { "code":  1583, "label": "د", "popup": {
+        "main": {"code": 1584, "label": "ذ" }
+      } },
       { "code":  -255, "label": "وو" },
       { "code":  1608, "label": "و" },
       { "code":  1734, "label": "ۆ" },
@@ -55,8 +55,10 @@
 
     ],
     [
-      { "code": 1600, "label": "kashida", "variation": "normal" },
-      { "code":  1574, "label": "ﺋ"},
+
+      { "code":  1574, "label": "ﺋ", "popup": {
+        "main": { "code": 1569, "label": "ء" }
+      } },
 
       { "code":  1662, "label": "پ", "popup": {
         "main": { "code": 1579, "label": "ث" }

app/src/main/assets/ime/text/characters/mod/jis.json

@@ -0,0 +1,29 @@
+{
+  "type": "characters/mod",
+  "name": "jis",
+  "label": "JIS",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code":     0, "type": "placeholder" },
+      { "code":    -5, "label": "delete", "type": "enter_editing" }
+    ],
+    [
+      { "code":  -202, "label": "view_symbols", "type": "system_gui" },
+      { "$": "char_width_selector",
+        "full": { "code": 12289, "label": "、", "groupId": 1 },
+        "half": { "code": 65380, "label": "､", "groupId": 1 }
+      },
+      { "code":  -210, "label": "language_switch", "type": "system_gui" },
+      { "code":  -213, "label": "switch_to_media_context", "type": "system_gui" },
+      { "code": 12288, "label": "空白" },
+      { "code": -9710, "label": "かな", "groupId": 97, "type": "system_gui" },
+      { "$": "char_width_selector",
+        "full": { "code": 12290, "label": "。", "groupId": 2 },
+        "half": { "code": 65377, "label": "｡", "groupId": 2 }
+      },
+      { "code":    10, "label": "enter", "groupId": 3, "type": "enter_editing" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/characters/mod/kurdish.json

@@ -6,14 +6,17 @@
   "direction": "rtl",
   "arrangement": [
     [
+      { "code": 1600, "label": "kashida", "popup":
+      { "main": { "code": 8204, "label": "half_space" }
+      } },
       { "code":    0, "type": "placeholder" },
       { "code":   -5, "label": "delete", "type": "enter_editing" }
     ],
     [
       { "code": -202, "label": "view_symbols", "type": "system_gui" },
       { "$": "variation_selector",
-        "default":  { "code": 1567, "label": "؟", "groupId": 1 },
-        "password": { "code": 1548, "label": "،", "groupId": 1 },
+        "default":  { "code": 1548, "label": "،", "groupId": 1 },
+        "password": { "code": 35, "label": "#", "groupId": 1 },
         "email":    { "code":   64, "label": "@", "groupId": 1 },
         "uri":      { "code":   47, "label": "/", "groupId": 1 }
       },

app/src/main/assets/ime/text/characters/nalmy.json

@@ -0,0 +1,41 @@
+{
+  "type": "characters",
+  "name": "nalmy",
+  "label": "NALMY",
+  "authors": [ "jeremiah-mille", "jasmcole" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "$": "auto_text_key", "code":  122, "label": "z" },
+      { "$": "auto_text_key", "code":  118, "label": "v" },
+      { "$": "auto_text_key", "code":  100, "label": "d" },
+      { "$": "auto_text_key", "code":  110, "label": "n" },
+      { "$": "auto_text_key", "code":   97, "label": "a" },
+      { "$": "auto_text_key", "code":  108, "label": "l" },
+      { "$": "auto_text_key", "code":  109, "label": "m" },
+      { "$": "auto_text_key", "code":  121, "label": "y" },
+      { "$": "auto_text_key", "code":  120, "label": "x" },
+      { "$": "auto_text_key", "code":  106, "label": "j" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  107, "label": "k" },
+      { "$": "auto_text_key", "code":  103, "label": "g" },
+      { "$": "auto_text_key", "code":  105, "label": "i" },
+      { "$": "auto_text_key", "code":  101, "label": "e" },
+      { "$": "auto_text_key", "code":  114, "label": "r" },
+      { "$": "auto_text_key", "code":  111, "label": "o" },
+      { "$": "auto_text_key", "code":  112, "label": "p" },
+      { "$": "auto_text_key", "code":   98, "label": "b" },
+      { "$": "auto_text_key", "code":  113, "label": "q" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  119, "label": "w" },
+      { "$": "auto_text_key", "code":  104, "label": "h" },
+      { "$": "auto_text_key", "code":  116, "label": "t" },
+      { "$": "auto_text_key", "code":  115, "label": "s" },
+      { "$": "auto_text_key", "code":   99, "label": "c" },
+      { "$": "auto_text_key", "code":  117, "label": "u" },
+      { "$": "auto_text_key", "code":  102, "label": "f" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/characters/sangaline.json

@@ -0,0 +1,41 @@
+{
+  "type": "characters",
+  "name": "sangaline",
+  "label": "Sangaline",
+  "authors": [ "jeremiah-miller", "sangaline" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "$": "auto_text_key", "code":  100, "label": "d" },
+      { "$": "auto_text_key", "code":  103, "label": "g" },
+      { "$": "auto_text_key", "code":  104, "label": "h" },
+      { "$": "auto_text_key", "code":  112, "label": "p" },
+      { "$": "auto_text_key", "code":   97, "label": "a" },
+      { "$": "auto_text_key", "code":  115, "label": "s" },
+      { "$": "auto_text_key", "code":  106, "label": "j" },
+      { "$": "auto_text_key", "code":  114, "label": "r" },
+      { "$": "auto_text_key", "code":  107, "label": "k" },
+      { "$": "auto_text_key", "code":  110, "label": "n" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  105, "label": "i" },
+      { "$": "auto_text_key", "code":  113, "label": "q" },
+      { "$": "auto_text_key", "code":  118, "label": "v" },
+      { "$": "auto_text_key", "code":  117, "label": "u" },
+      { "$": "auto_text_key", "code":  119, "label": "w" },
+      { "$": "auto_text_key", "code":   99, "label": "c" },
+      { "$": "auto_text_key", "code":  108, "label": "l" },
+      { "$": "auto_text_key", "code":  120, "label": "x" },
+      { "$": "auto_text_key", "code":  109, "label": "m" }
+    ],
+    [
+      { "$": "auto_text_key", "code":  116, "label": "t" },
+      { "$": "auto_text_key", "code":  121, "label": "y" },
+      { "$": "auto_text_key", "code":   98, "label": "b" },
+      { "$": "auto_text_key", "code":  101, "label": "e" },
+      { "$": "auto_text_key", "code":  122, "label": "z" },
+      { "$": "auto_text_key", "code":  102, "label": "f" },
+      { "$": "auto_text_key", "code":  111, "label": "o" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/characters/turkish_f.json

@@ -9,7 +9,10 @@
       { "$": "auto_text_key", "code":  102, "label": "f" },
       { "$": "auto_text_key", "code":  103, "label": "g" },
       { "$": "auto_text_key", "code":  287, "label": "ğ" },
-      { "$": "auto_text_key", "code":  305, "label": "ı" },
+      { "$": "case_selector",
+        "lower": { "code":  305, "label": "ı" },
+        "upper": { "code":   73, "label": "I" }
+      },
       { "$": "auto_text_key", "code":  111, "label": "o" },
       { "$": "auto_text_key", "code":  100, "label": "d" },
       { "$": "auto_text_key", "code":  114, "label": "r" },
@@ -21,7 +24,10 @@
     ],
     [
       { "$": "auto_text_key", "code":  117, "label": "u" },
-      { "$": "auto_text_key", "code":  105, "label": "i" },
+      { "$": "case_selector",
+        "lower": { "code":  105, "label": "i" },
+        "upper": { "code":  304, "label": "İ" }
+      },
       { "$": "auto_text_key", "code":  101, "label": "e" },
       { "$": "auto_text_key", "code":   97, "label": "a" },
       { "$": "auto_text_key", "code":  252, "label": "ü" },

app/src/main/assets/ime/text/characters/turkish_q.json

@@ -13,7 +13,10 @@
       { "$": "auto_text_key", "code":  116, "label": "t" },
       { "$": "auto_text_key", "code":  121, "label": "y" },
       { "$": "auto_text_key", "code":  117, "label": "u" },
-      { "$": "auto_text_key", "code":  305, "label": "ı" },
+      { "$": "case_selector",
+        "lower": { "code":  305, "label": "ı" },
+        "upper": { "code":   73, "label": "I" }
+      },
       { "$": "auto_text_key", "code":  111, "label": "o" },
       { "$": "auto_text_key", "code":  112, "label": "p" },
       { "$": "auto_text_key", "code":  287, "label": "ğ" },
@@ -30,7 +33,10 @@
       { "$": "auto_text_key", "code":  107, "label": "k" },
       { "$": "auto_text_key", "code":  108, "label": "l" },
       { "$": "auto_text_key", "code":  351, "label": "ş" },
-      { "$": "auto_text_key", "code":  105, "label": "i" }
+      { "$": "case_selector",
+        "lower": { "code":  105, "label": "i" },
+        "upper": { "code":  304, "label": "İ" }
+      }
     ],
     [
       { "$": "auto_text_key", "code":  122, "label": "z" },

app/src/main/assets/ime/text/numeric/advanced/western_arabic_pc.json

@@ -0,0 +1,55 @@
+{
+  "type": "numeric_advanced",
+  "name": "western_arabic_pc",
+  "label": "Western Arabic (PC)",
+  "authors": [ "patrickgold" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code":   43, "label": "+", "popup": {
+        "relevant": [
+          { "code":   45, "label": "-" },
+          { "code":   42, "label": "*" },
+          { "code":   47, "label": "/" }
+        ]
+      } },
+      { "code":   55, "label": "7", "type": "numeric" },
+      { "code":   56, "label": "8", "type": "numeric" },
+      { "code":   57, "label": "9", "type": "numeric" },
+      { "code":   37, "label": "%" }
+    ],
+    [
+      { "code":   40, "label": "(", "popup": {
+        "relevant": [
+          { "code":   91, "label": "[" },
+          { "code":  123, "label": "{" }
+        ]
+      } },
+      { "code":   52, "label": "4", "type": "numeric" },
+      { "code":   53, "label": "5", "type": "numeric" },
+      { "code":   54, "label": "6", "type": "numeric" },
+      { "code":   32, "label": "space" }
+    ],
+    [
+      { "code":   41, "label": ")", "popup": {
+        "relevant": [
+          { "code":   93, "label": "]" },
+          { "code":  125, "label": "}" }
+        ]
+      } },
+      { "code":   49, "label": "1", "type": "numeric" },
+      { "code":   50, "label": "2", "type": "numeric" },
+      { "code":   51, "label": "3", "type": "numeric" },
+      { "code":   -5, "label": "delete", "type": "enter_editing" }
+    ],
+    [
+      { "code": -201, "label": "view_characters", "type": "system_gui" },
+      { "code":   44, "label": "," },
+      { "code": -202, "label": "view_symbols", "type": "system_gui" },
+      { "code":   48, "label": "0", "type": "numeric" },
+      { "code":   61, "label": "=" },
+      { "code":   46, "label": "." },
+      { "code":   10, "label": "enter", "groupId": 3, "type": "enter_editing" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/numeric/row/cjk.json

@@ -0,0 +1,108 @@
+{
+  "type": "numeric_row",
+  "name": "cjk",
+  "label": "CJK",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code": 19968, "label": "一", "type": "numeric", "popup": {
+        "main": { "code":   49, "label": "1" },
+        "relevant": [
+          { "code": 22769, "label": "壱" },
+          { "code": 22777, "label": "壹" },
+          { "code": 24332, "label": "弌" },
+          { "code": 65297, "label": "１" }
+        ]
+      } },
+      { "code": 20108, "label": "二", "type": "numeric", "popup": {
+        "main": { "code":   50, "label": "2" },
+        "relevant": [
+          { "code": 24336, "label": "弐" },
+          { "code": 36019, "label": "貳" },
+          { "code": 36014, "label": "貮" },
+          { "code": 65298, "label": "２" }
+        ]
+      } },
+      { "code": 19977, "label": "三", "type": "numeric", "popup": {
+        "main": { "code":   51, "label": "3" },
+        "relevant": [
+          { "code": 21442, "label": "参" },
+          { "code": 21443, "label": "參" },
+          { "code": 24334, "label": "弎" },
+          { "code": 65299, "label": "３" }
+        ]
+      } },
+      { "code": 22235, "label": "四", "type": "numeric", "popup": {
+        "main": { "code":   52, "label": "4" },
+        "relevant": [
+          { "code": 32902, "label": "肆" },
+          { "code": 18825, "label": "䦉" },
+          { "code": 20118, "label": "亖" },
+          { "code": 65300, "label": "３" }
+        ]
+      } },
+      { "code": 20116, "label": "五", "type": "numeric", "popup": {
+        "main": { "code":   53, "label": "5" },
+        "relevant": [
+          { "code": 20237, "label": "伍" },
+          { "code": 65301, "label": "５" }
+        ]
+      } },
+      { "code": 20845, "label": "六", "type": "numeric", "popup": {
+        "main": { "code":   54, "label": "6" },
+        "relevant": [
+          { "code": 38520, "label": "陸" },
+          { "code": 65302, "label": "６" }
+        ]
+      } },
+      { "code": 19971, "label": "七", "type": "numeric", "popup": {
+        "main": { "code":   55, "label": "7" },
+        "relevant": [
+          { "code": 28422, "label": "漆" },
+          { "code": 26578, "label": "柒" },
+          { "code": 65303, "label": "７" }
+        ]
+      } },
+      { "code": 20843, "label": "八", "type": "numeric", "popup": {
+        "main": { "code":   56, "label": "8" },
+        "relevant": [
+          { "code": 25420, "label": "捌" },
+          { "code": 65304, "label": "８" }
+        ]
+      } },
+      { "code": 20061, "label": "九", "type": "numeric", "popup": {
+        "main": {"code":   57, "label": "9" },
+        "relevant": [
+          { "code": 29590, "label": "玖" },
+          { "code": 65305, "label": "９" }
+        ]
+      } },
+      { "code": 38646, "label": "零", "type": "numeric", "popup": {
+        "main": { "code":   48, "label": "0" },
+        "relevant": [
+          { "code": 12295, "label": "〇" },
+          { "code": 65296, "label": "０" }
+        ]
+      } },
+      { "code": 21313, "label": "十", "type": "numeric", "popup": {
+        "main": { "code": 25342, "label": "拾" },
+        "relevant": [
+          { "code": 20160, "label": "什" }
+        ]
+      } },
+      { "code": 30334, "label": "百", "type": "numeric", "popup": {
+        "main": { "code": 20336, "label": "佰" },
+        "relevant": [
+          { "code": 38476, "label": "陌" }
+        ]
+      } },
+      { "code": 21315, "label": "千", "type": "numeric", "popup": {
+        "main": { "code": 20191, "label": "仟" },
+        "relevant": [
+          { "code": 38433, "label": "阡" }
+        ]
+      } }
+    ]
+  ]
+}

app/src/main/assets/ime/text/numeric/western_arabic_pc.json

@@ -0,0 +1,35 @@
+{
+  "type": "numeric",
+  "name": "western_arabic_pc",
+  "label": "Western Arabic (PC)",
+  "authors": [ "patrickgold" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code":   55, "label": "7", "type": "numeric" },
+      { "code":   56, "label": "8", "type": "numeric" },
+      { "code":   57, "label": "9", "type": "numeric" },
+      { "code":   45, "label": "-" }
+    ],
+    [
+      { "code":   52, "label": "4", "type": "numeric" },
+      { "code":   53, "label": "5", "type": "numeric" },
+      { "code":   54, "label": "6", "type": "numeric" },
+      { "code":   32, "label": "space" }
+    ],
+    [
+      { "code":   49, "label": "1", "type": "numeric" },
+      { "code":   50, "label": "2", "type": "numeric" },
+      { "code":   51, "label": "3", "type": "numeric" },
+      { "code":   -5, "label": "delete", "type": "enter_editing" }
+    ],
+    [
+      { "code":   44, "label": "," },
+      { "code":   48, "label": "0", "type": "numeric", "popup": {
+        "main": { "code":   43, "label": "+" }
+      } },
+      { "code":   46, "label": "." },
+      { "code":   10, "label": "enter", "type": "enter_editing" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/symbols/cjk.json

@@ -0,0 +1,283 @@
+{
+  "type": "symbols",
+  "name": "cjk",
+  "label": "CJK",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "modifier": "cjk",
+  "arrangement": [
+    [
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65312, "label": "＠", "popup": {
+            "main": { "code": 64, "label": "@" }
+          }
+        },
+        "half": { "code": 64, "label": "@" }
+      },
+      { "code": 12306, "label": "〒", "popup": {
+          "main": { "code": 12320, "label": "〠" }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65283, "label": "＃", "popup": {
+            "main": { "code": 35, "label": "#" },
+            "relevant": [
+              { "code": 8470, "label": "№" }
+            ]
+          }
+        },
+        "half": { "code": 35, "label": "#", "popup": {
+            "main": { "code": 65283, "label": "＃" },
+            "relevant": [
+              { "code": 8470, "label": "№" }
+            ]
+          }
+        }
+      },
+      { "code": -801, "label": "currency_slot_1", "popup": {
+          "main": { "code": -802, "label": "currency_slot_2" },
+          "relevant": [
+            { "code": -806, "label": "currency_slot_6" },
+            { "code": -803, "label": "currency_slot_3" },
+            { "code": -804, "label": "currency_slot_4" },
+            { "code": -805, "label": "currency_slot_5" }
+          ]
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65285, "label": "％", "popup": {
+            "main": { "code": 37, "label": "%" },
+            "relevant": [
+              { "code": 8240, "label": "‰" },
+              { "code": 8453, "label": "℅" }
+            ]
+          }
+        },
+        "half": { "code": 37, "label": "%", "popup": {
+            "main": { "code": 65285, "label": "％" },
+            "relevant": [
+              { "code": 8240, "label": "‰" },
+              { "code": 8453, "label": "℅" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65286, "label": "＆", "popup": {
+            "main": { "code": 38, "label": "&" }
+          }
+        },
+        "half": { "code": 38, "label": "&", "popup": {
+            "main": { "code": 65286, "label": "＆" }
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65293, "label": "－", "popup": {
+            "main": { "code": 65343, "label": "＿" },
+            "relevant": [
+              { "code": 45, "label": "-" },
+              { "code": 95, "label": "_" },
+              { "code": 8212, "label": "—" },
+              { "code": 8211, "label": "–" },
+              { "code": 183, "label": "·" }
+            ]
+          }
+        },
+        "half": { "code": 45, "label": "-", "popup": {
+            "main": { "code": 95, "label": "_" },
+            "relevant": [
+              { "code": 65293, "label": "－" },
+              { "code": 65343, "label": "＿" },
+              { "code": 8212, "label": "—" },
+              { "code": 8211, "label": "–" },
+              { "code": 183, "label": "·" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65291, "label": "＋", "popup": {
+            "main": { "code": 43, "label": "+" },
+            "relevant": [
+              { "code": 177, "label": "±" }
+            ]
+          }
+        },
+        "half": { "code": 43, "label": "+", "popup": {
+            "main": { "code": 65291, "label": "＋" },
+            "relevant": [
+              { "code": 177, "label": "±" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 12300, "label": "「", "popup": {
+            "main": { "code": 12302, "label": "『" },
+            "relevant": [
+              { "code": 12304, "label": "【" },
+              { "code": 12310, "label": "〖" }
+            ]
+          }
+        },
+        "half": { "code": 65378, "label": "｢", "popup": {
+            "main": { "code": 12301, "label": "」" },
+            "relevant": [
+              { "code": 12303, "label": "』" },
+              { "code": 12304, "label": "【" },
+              { "code": 12310, "label": "〖" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 12301, "label": "」", "popup": {
+            "main": { "code": 12303, "label": "』" },
+            "relevant": [
+              { "code": 12305, "label": "】" },
+              { "code": 12311, "label": "〗" }
+            ]
+          }
+        },
+        "half": { "code": 65379, "label": "｣", "popup": {
+            "main": { "code": 12301, "label": "」" },
+            "relevant": [
+              { "code": 12303, "label": "』" },
+              { "code": 12305, "label": "】" },
+              { "code": 12311, "label": "〗" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65295, "label": "／", "popup": {
+            "main": { "code": 47, "label": "/" }
+          }
+        },
+        "half": { "code": 47, "label": "/", "popup": {
+            "main": { "code": 65295, "label": "／" }
+          }
+        }
+      }
+    ],
+    [
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65290, "label": "＊", "popup": {
+            "main": { "code": 8251, "label": "※" },
+            "relevant": [
+              { "code": 42, "label": "*" },
+              { "code": 8224, "label": "†" },
+              { "code": 9733, "label": "★" },
+              { "code": 8225, "label": "‡" }
+            ]
+          }
+        },
+        "half": {  "code": 42, "label": "*", "popup": {
+            "main": { "code": 65290, "label": "＊" },
+            "relevant": [
+              { "code": 8251, "label": "※" },
+              { "code": 8224, "label": "†" },
+              { "code": 9733, "label": "★" },
+              { "code": 8225, "label": "‡" }
+            ]
+          }
+        }
+      },
+      { "code": 34, "label": "\"", "popup": {
+          "main": { "code": 8221, "label": "”" },
+          "relevant": [
+            { "code": 8222, "label": "„" },
+            { "code": 8220, "label": "“" },
+            { "code": 171, "label": "«" },
+            { "code": 187, "label": "»" }
+          ]
+        }
+      },
+      { "code": 39, "label": "'", "popup": {
+          "main": { "code": 8217, "label": "’" },
+          "relevant": [
+            { "code": 8218, "label": "‚" },
+            { "code": 8216, "label": "‘" },
+            { "code": 8249, "label": "‹" },
+            { "code": 8250, "label": "›" }
+          ]
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65306, "label": "：", "popup": {
+            "main": { "code": 58, "label": ":" },
+            "relevant": [
+              { "code": 8942, "label": "⋮" }
+            ]
+          }
+        },
+        "half": { "code": 58, "label": ":", "popup": {
+            "main": { "code": 65306, "label": "：" },
+            "relevant": [
+              { "code": 8942, "label": "⋮" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65307, "label": "；", "popup": {
+            "main": { "code": 59, "label": ";" }
+          }
+        },
+        "half": { "code": 59, "label": ";", "popup": {
+            "main": { "code": 65307, "label": "；" }
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65281, "label": "！", "popup": {
+            "main": { "code": 33, "label": "!" },
+            "relevant": [
+              { "code": 161, "label": "¡" }
+            ]
+          }
+        },
+        "half": { "code": 33, "label": "!", "popup": {
+            "main": { "code": 65281, "label": "！" },
+            "relevant": [
+              { "code": 161, "label": "¡" }
+            ]
+          }
+        }
+      },
+      {
+        "$": "char_width_selector",
+        "full": { "code": 65311, "label": "？", "popup": {
+            "main": { "code": 63, "label": "?" },
+            "relevant": [
+              { "code": 191, "label": "¿" },
+              { "code": 8253, "label": "‽" }
+            ]
+          }
+        },
+        "half": {  "code": 63, "label": "?", "popup": {
+            "main": { "code": 65311, "label": "？" },
+            "relevant": [
+              { "code": 191, "label": "¿" },
+              { "code": 8253, "label": "‽" }
+            ]
+          }
+        }
+      }
+    ]
+  ]
+}

app/src/main/assets/ime/text/symbols/mod/cjk.json

@@ -0,0 +1,55 @@
+{
+  "type": "symbols/mod",
+  "name": "cjk",
+  "label": "CJK",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code": -203, "label": "view_symbols2", "type": "system_gui" },
+      { "code":    0, "type": "placeholder" },
+      { "code":   -5, "label": "delete", "type": "enter_editing" }
+    ],
+    [
+      { "code": -201, "label": "view_characters", "type": "system_gui" },
+      { "$": "char_width_selector",
+        "full": { "code": 12289, "label": "、", "popup": {
+            "main": { "code":   44, "label": "," }
+          }
+        },
+        "half": { "code": 65380, "label": "､", "popup": {
+            "main": { "code":   44, "label": "," }
+          }
+        }
+      },
+      { "code": -205, "label": "view_numeric_advanced", "type": "system_gui" },
+      { "code": 12288, "label": "空白" },
+      { "code": -9701, "label": "char_width_switcher", "type": "system_gui", "popup": {
+          "relevant": [
+            { "code": -9702, "label": "char_width_full", "type": "system_gui" },
+            { "code": -9703, "label": "char_width_half", "type": "system_gui" }
+          ]
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 12290, "label": "。", "popup": {
+            "main": { "code": 8230, "label": "…" },
+            "relevant": [
+              { "code": 12539, "label": "・" },
+              { "code":    46, "label": "." }
+            ]
+          }
+        },
+        "half": { "code": 65377, "label": "｡", "popup": {
+            "main": { "code": 8230, "label": "…" },
+            "relevant": [
+              { "code": 65381, "label": "･" },
+              { "code":    46, "label": "." }
+            ]
+          }
+        }
+      },
+      { "code":   10, "label": "enter", "groupId": 3, "type": "enter_editing" }
+    ]
+  ]
+}

app/src/main/assets/ime/text/symbols2/cjk.json

@@ -0,0 +1,211 @@
+{
+  "type": "symbols2",
+  "name": "cjk",
+  "label": "CJK",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "modifier": "cjk",
+  "arrangement": [
+    [
+      { "$": "char_width_selector",
+        "full": { "code": 12316, "label": "〜", "popup": {
+          "main": { "code":  126, "label": "~" }
+          }
+        },
+        "half": { "code":  126, "label": "~", "popup": {
+          "main": { "code": 12316, "label": "〜" }
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65344, "label": "｀", "popup": {
+          "main": { "code":   96, "label": "`" }
+          }
+        },
+        "half": { "code":   96, "label": "`", "popup": {
+          "main": { "code": 65344, "label": "｀" }
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65372, "label": "｜", "popup": {
+          "main": { "code":  124, "label": "|" }
+          }
+        },
+        "half": { "code":  124, "label": "|", "popup": {
+          "main": { "code": 65372, "label": "｜" }
+          }
+        }
+      },
+      { "code": 12539, "label": "・", "popup": {
+        "main": { "code": 9834, "label": "♪" },
+        "relevant": [
+          { "code": 8226, "label": "•" },
+          { "code": 9827, "label": "♣" },
+          { "code": 9824, "label": "♠" },
+          { "code": 9829, "label": "♥" },
+          { "code": 9830, "label": "♦" }
+        ]
+      } },
+      { "code": 8730, "label": "√" },
+      { "code":  960, "label": "π", "popup": {
+        "main": { "code":  928, "label": "Π" },
+        "relevant": [
+          { "code":  969, "label": "ω" },
+          { "code":  945, "label": "α" },
+          { "code":  946, "label": "β" },
+          { "code":  937, "label": "Ω" },
+          { "code":  956, "label": "μ" }
+        ]
+      } },
+      { "code":  247, "label": "÷" },
+      { "code":  215, "label": "×" },
+      { "code":  182, "label": "¶" },
+      { "code": 8710, "label": "∆" }
+    ],
+    [
+      { "code": -805, "label": "currency_slot_5" },
+      { "code": -804, "label": "currency_slot_4" },
+      { "code": -803, "label": "currency_slot_3" },
+      { "code": -802, "label": "currency_slot_2" },
+      { "code":   94, "label": "^", "popup": {
+        "main": { "code": 8593, "label": "↑" },
+        "relevant": [
+          { "code": 8592, "label": "←" },
+          { "code": 8595, "label": "↓" },
+          { "code": 8594, "label": "→" }
+        ]
+      } },
+      { "code":  176, "label": "°", "popup": {
+        "main": { "code": 8242, "label": "′" },
+        "relevant": [
+          { "code": 8243, "label": "″" }
+        ]
+      } },
+      { "$": "char_width_selector",
+        "full":
+          { "code": 65309, "label": "＝", "popup": {
+            "main": { "code": 8800, "label": "≠" },
+            "relevant": [
+              { "code":   61, "label": "=" },
+              { "code": 8734, "label": "∞" },
+              { "code": 8776, "label": "≈" }
+            ]
+          }
+        },
+        "half":
+          { "code":   61, "label": "=", "popup": {
+            "main": { "code": 8800, "label": "≠" },
+            "relevant": [
+              { "code":   61, "label": "=" },
+              { "code": 8734, "label": "∞" },
+              { "code": 8776, "label": "≈" }
+            ]
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65371, "label": "｛", "popup": {
+            "main": { "code": 65288, "label": "（" },
+            "relevant": [
+              { "code":   123, "label": "{" },
+              { "code": 12308, "label": "〔" },
+              { "code": 12312, "label": "〘" },
+              { "code": 65375, "label": "｟" }
+            ]
+          }
+        },
+        "half": { "code":   123, "label": "{", "popup": {
+            "main": { "code": 40, "label": "(" },
+            "relevant": [
+              { "code": 65371, "label": "｛" }
+            ]
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65373, "label": "｝", "popup": {
+            "main": { "code": 65289, "label": "）" },
+            "relevant": [
+              { "code":   125, "label": "}" },
+              { "code": 12309, "label": "〕" },
+              { "code": 12313, "label": "〙" },
+              { "code": 65376, "label": "｠" }
+            ]
+          }
+        },
+        "half": { "code":   125, "label": "}", "popup": {
+            "main": { "code": 41, "label": ")" },
+            "relevant": [
+              { "code": 65373, "label": "｝" }
+            ]
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65340, "label": "＼", "popup": {
+            "main": { "code":   92, "label": "\\" }
+          }
+        },
+        "half": { "code":   92, "label": "\\", "popup": {
+            "main": { "code": 65340, "label": "＼" }
+          }
+        }
+      }
+    ],
+    [
+      { "code": 12292, "label": "〄" },
+      { "code": 12293, "label": "々" },
+      { "code": 12294, "label": "〆" },
+      { "code": 12295, "label": "〇" },
+      { "$": "kana_selector",
+        "hira": { "code": 12445, "label": "ゝ", "popup": {
+              "main": { "code": 12446, "label": "ゞ" },
+              "relevant": [
+                { "code": 12541, "label": "ヽ" },
+                { "code": 12542, "label": "ヾ" }
+              ]
+          }
+        },
+        "kata": { "code": 12541, "label": "ヽ", "popup": {
+              "main": { "code": 12542, "label": "ヾ" },
+              "relevant": [
+                { "code": 12445, "label": "ゝ" },
+                { "code": 12446, "label": "ゞ" }
+              ]
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65339, "label": "［", "popup": {
+            "main": { "code":   91, "label": "[" },
+            "relevant": [
+              { "code": 12314, "label": "〚" },
+              { "code": 12304, "label": "【" },
+              { "code": 12310, "label": "〖" }
+            ]
+          }
+        },
+        "half": { "code":   91, "label": "[", "popup": {
+            "main": { "code": 65339, "label": "［" }
+          }
+        }
+      },
+      { "$": "char_width_selector",
+        "full": { "code": 65341, "label": "］", "popup": {
+            "main": { "code":   93, "label": "]" },
+            "relevant": [
+              { "code": 12315, "label": "〛" },
+              { "code": 12305, "label": "】" },
+              { "code": 12311, "label": "〗" }
+            ]
+          }
+        },
+        "half": { "code":   93, "label": "]", "popup": {
+            "main": { "code": 65341, "label": "］" }
+          }
+        }
+      }
+    ]
+  ]
+}

app/src/main/assets/ime/text/symbols2/mod/cjk.json

@@ -0,0 +1,45 @@
+{
+  "type": "symbols2/mod",
+  "name": "cjk",
+  "label": "CJK",
+  "authors": [ "waelwindows" ],
+  "direction": "ltr",
+  "arrangement": [
+    [
+      { "code": -202, "label": "view_symbols", "type": "system_gui" },
+      { "code":    0, "type": "placeholder" },
+      { "code":   -5, "label": "delete", "type": "enter_editing" }
+    ],
+    [
+      { "code": -201, "label": "view_characters", "type": "system_gui" },
+      { "code": 12296, "label": "〈", "popup": {
+        "main": { "code": 12298, "label": "《" },
+        "relevant": [
+          { "code":  8804, "label": "≤" },
+          { "code":  8249, "label": "‹" },
+          { "code": 10216, "label": "⟨" },
+          { "code": 65308, "label": "＜" }
+        ]
+      } },
+      { "code": -205, "label": "view_numeric_advanced", "type": "system_gui" },
+      { "code": 12288, "label": "空白" },
+      { "code": -9701, "label": "char_width_switcher", "type": "system_gui", "popup": {
+          "relevant": [
+            { "code": -9702, "label": "char_width_full", "type": "system_gui" },
+            { "code": -9703, "label": "char_width_half", "type": "system_gui" }
+          ]
+        }
+      },
+      { "code": 12297, "label": "〉", "popup": {
+        "main": { "code": 12299, "label": "》" },
+        "relevant": [
+          { "code":    62, "label": ">" },
+          { "code":  8805, "label": "≥" },
+          { "code": 10217, "label": "⟩" },
+          { "code": 65310, "label": "＞" }
+        ]
+      } },
+      { "code":   10, "label": "enter", "groupId": 3, "type": "enter_editing" }
+    ]
+  ]
+}

app/src/main/assets/license/open_source_licenses.html

@@ -294,6 +294,422 @@ SOFTWARE.
 
     <hr>
 
+    <h3>ICU4C: International Components for Unicode</h3>
+    <span>Copyright © 1991-2020 Unicode, Inc. All rights reserved.</span>
+    <pre>
+Distributed under the Terms of Use in https://www.unicode.org/copyright.html.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of the Unicode data files and any associated documentation
+(the "Data Files") or Unicode software and any associated documentation
+(the "Software") to deal in the Data Files or Software
+without restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, and/or sell copies of
+the Data Files or Software, and to permit persons to whom the Data Files
+or Software are furnished to do so, provided that either
+(a) this copyright and permission notice appear with all copies
+of the Data Files or Software, or
+(b) this copyright and permission notice appear in associated
+Documentation.
+
+THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF
+ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT OF THIRD PARTY RIGHTS.
+IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS
+NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL
+DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
+DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
+TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+PERFORMANCE OF THE DATA FILES OR SOFTWARE.
+
+Except as contained in this notice, the name of a copyright holder
+shall not be used in advertising or otherwise to promote the sale,
+use or other dealings in these Data Files or Software without prior
+written authorization of the copyright holder.
+
+---------------------
+
+Third-Party Software Licenses
+
+This section contains third-party software notices and/or additional
+terms for licensed third-party software components included within ICU
+libraries.
+
+1. ICU License - ICU 1.8.1 to ICU 57.1
+
+COPYRIGHT AND PERMISSION NOTICE
+
+Copyright (c) 1995-2016 International Business Machines Corporation and others
+All rights reserved.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, and/or sell copies of the Software, and to permit persons
+to whom the Software is furnished to do so, provided that the above
+copyright notice(s) and this permission notice appear in all copies of
+the Software and that both the above copyright notice(s) and this
+permission notice appear in supporting documentation.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
+OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY
+SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER
+RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF
+CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
+CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+Except as contained in this notice, the name of a copyright holder
+shall not be used in advertising or otherwise to promote the sale, use
+or other dealings in this Software without prior written authorization
+of the copyright holder.
+
+All trademarks and registered trademarks mentioned herein are the
+property of their respective owners.
+
+2. Chinese/Japanese Word Break Dictionary Data (cjdict.txt)
+
+ #     The Google Chrome software developed by Google is licensed under
+ # the BSD license. Other software included in this distribution is
+ # provided under other licenses, as set forth below.
+ #
+ #  The BSD License
+ #  http://opensource.org/licenses/bsd-license.php
+ #  Copyright (C) 2006-2008, Google Inc.
+ #
+ #  All rights reserved.
+ #
+ #  Redistribution and use in source and binary forms, with or without
+ # modification, are permitted provided that the following conditions are met:
+ #
+ #  Redistributions of source code must retain the above copyright notice,
+ # this list of conditions and the following disclaimer.
+ #  Redistributions in binary form must reproduce the above
+ # copyright notice, this list of conditions and the following
+ # disclaimer in the documentation and/or other materials provided with
+ # the distribution.
+ #  Neither the name of  Google Inc. nor the names of its
+ # contributors may be used to endorse or promote products derived from
+ # this software without specific prior written permission.
+ #
+ #
+ #  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ # CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ # INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ # MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ # BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ # LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+ # NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ # SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ #
+ #
+ #  The word list in cjdict.txt are generated by combining three word lists
+ # listed below with further processing for compound word breaking. The
+ # frequency is generated with an iterative training against Google web
+ # corpora.
+ #
+ #  * Libtabe (Chinese)
+ #    - https://sourceforge.net/project/?group_id=1519
+ #    - Its license terms and conditions are shown below.
+ #
+ #  * IPADIC (Japanese)
+ #    - http://chasen.aist-nara.ac.jp/chasen/distribution.html
+ #    - Its license terms and conditions are shown below.
+ #
+ #  ---------COPYING.libtabe ---- BEGIN--------------------
+ #
+ #  /*
+ #   * Copyright (c) 1999 TaBE Project.
+ #   * Copyright (c) 1999 Pai-Hsiang Hsiao.
+ #   * All rights reserved.
+ #   *
+ #   * Redistribution and use in source and binary forms, with or without
+ #   * modification, are permitted provided that the following conditions
+ #   * are met:
+ #   *
+ #   * . Redistributions of source code must retain the above copyright
+ #   *   notice, this list of conditions and the following disclaimer.
+ #   * . Redistributions in binary form must reproduce the above copyright
+ #   *   notice, this list of conditions and the following disclaimer in
+ #   *   the documentation and/or other materials provided with the
+ #   *   distribution.
+ #   * . Neither the name of the TaBE Project nor the names of its
+ #   *   contributors may be used to endorse or promote products derived
+ #   *   from this software without specific prior written permission.
+ #   *
+ #   * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ #   * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ #   * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ #   * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ #   * REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ #   * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ #   * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ #   * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ #   * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ #   * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ #   * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ #   * OF THE POSSIBILITY OF SUCH DAMAGE.
+ #   */
+ #
+ #  /*
+ #   * Copyright (c) 1999 Computer Systems and Communication Lab,
+ #   *                    Institute of Information Science, Academia
+ #       *                    Sinica. All rights reserved.
+ #   *
+ #   * Redistribution and use in source and binary forms, with or without
+ #   * modification, are permitted provided that the following conditions
+ #   * are met:
+ #   *
+ #   * . Redistributions of source code must retain the above copyright
+ #   *   notice, this list of conditions and the following disclaimer.
+ #   * . Redistributions in binary form must reproduce the above copyright
+ #   *   notice, this list of conditions and the following disclaimer in
+ #   *   the documentation and/or other materials provided with the
+ #   *   distribution.
+ #   * . Neither the name of the Computer Systems and Communication Lab
+ #   *   nor the names of its contributors may be used to endorse or
+ #   *   promote products derived from this software without specific
+ #   *   prior written permission.
+ #   *
+ #   * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ #   * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ #   * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ #   * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ #   * REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ #   * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ #   * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ #   * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ #   * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ #   * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ #   * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ #   * OF THE POSSIBILITY OF SUCH DAMAGE.
+ #   */
+ #
+ #  Copyright 1996 Chih-Hao Tsai @ Beckman Institute,
+ #      University of Illinois
+ #  c-tsai4@uiuc.edu  http://casper.beckman.uiuc.edu/~c-tsai4
+ #
+ #  ---------------COPYING.libtabe-----END--------------------------------
+ #
+ #
+ #  ---------------COPYING.ipadic-----BEGIN-------------------------------
+ #
+ #  Copyright 2000, 2001, 2002, 2003 Nara Institute of Science
+ #  and Technology.  All Rights Reserved.
+ #
+ #  Use, reproduction, and distribution of this software is permitted.
+ #  Any copy of this software, whether in its original form or modified,
+ #  must include both the above copyright notice and the following
+ #  paragraphs.
+ #
+ #  Nara Institute of Science and Technology (NAIST),
+ #  the copyright holders, disclaims all warranties with regard to this
+ #  software, including all implied warranties of merchantability and
+ #  fitness, in no event shall NAIST be liable for
+ #  any special, indirect or consequential damages or any damages
+ #  whatsoever resulting from loss of use, data or profits, whether in an
+ #  action of contract, negligence or other tortuous action, arising out
+ #  of or in connection with the use or performance of this software.
+ #
+ #  A large portion of the dictionary entries
+ #  originate from ICOT Free Software.  The following conditions for ICOT
+ #  Free Software applies to the current dictionary as well.
+ #
+ #  Each User may also freely distribute the Program, whether in its
+ #  original form or modified, to any third party or parties, PROVIDED
+ #  that the provisions of Section 3 ("NO WARRANTY") will ALWAYS appear
+ #  on, or be attached to, the Program, which is distributed substantially
+ #  in the same form as set out herein and that such intended
+ #  distribution, if actually made, will neither violate or otherwise
+ #  contravene any of the laws and regulations of the countries having
+ #  jurisdiction over the User or the intended distribution itself.
+ #
+ #  NO WARRANTY
+ #
+ #  The program was produced on an experimental basis in the course of the
+ #  research and development conducted during the project and is provided
+ #  to users as so produced on an experimental basis.  Accordingly, the
+ #  program is provided without any warranty whatsoever, whether express,
+ #  implied, statutory or otherwise.  The term "warranty" used herein
+ #  includes, but is not limited to, any warranty of the quality,
+ #  performance, merchantability and fitness for a particular purpose of
+ #  the program and the nonexistence of any infringement or violation of
+ #  any right of any third party.
+ #
+ #  Each user of the program will agree and understand, and be deemed to
+ #  have agreed and understood, that there is no warranty whatsoever for
+ #  the program and, accordingly, the entire risk arising from or
+ #  otherwise connected with the program is assumed by the user.
+ #
+ #  Therefore, neither ICOT, the copyright holder, or any other
+ #  organization that participated in or was otherwise related to the
+ #  development of the program and their respective officials, directors,
+ #  officers and other employees shall be held liable for any and all
+ #  damages, including, without limitation, general, special, incidental
+ #  and consequential damages, arising out of or otherwise in connection
+ #  with the use or inability to use the program or any product, material
+ #  or result produced or otherwise obtained by using the program,
+ #  regardless of whether they have been advised of, or otherwise had
+ #  knowledge of, the possibility of such damages at any time during the
+ #  project or thereafter.  Each user will be deemed to have agreed to the
+ #  foregoing by his or her commencement of use of the program.  The term
+ #  "use" as used herein includes, but is not limited to, the use,
+ #  modification, copying and distribution of the program and the
+ #  production of secondary products from the program.
+ #
+ #  In the case where the program, whether in its original form or
+ #  modified, was distributed or delivered to or received by a user from
+ #  any person, organization or entity other than ICOT, unless it makes or
+ #  grants independently of ICOT any specific warranty to the user in
+ #  writing, such person, organization or entity, will also be exempted
+ #  from and not be held liable to the user for any such damages as noted
+ #  above as far as the program is concerned.
+ #
+ #  ---------------COPYING.ipadic-----END----------------------------------
+
+3. Lao Word Break Dictionary Data (laodict.txt)
+
+ # Copyright (C) 2016 and later: Unicode, Inc. and others.
+ # License & terms of use: http://www.unicode.org/copyright.html
+ # Copyright (c) 2015 International Business Machines Corporation
+ # and others. All Rights Reserved.
+ #
+ # Project: https://github.com/rober42539/lao-dictionary
+ # Dictionary: https://github.com/rober42539/lao-dictionary/laodict.txt
+ # License: https://github.com/rober42539/lao-dictionary/LICENSE.txt
+ #          (copied below)
+ #
+ #	This file is derived from the above dictionary version of Nov 22, 2020
+ #  ----------------------------------------------------------------------
+ #  Copyright (C) 2013 Brian Eugene Wilson, Robert Martin Campbell.
+ #  All rights reserved.
+ #
+ #  Redistribution and use in source and binary forms, with or without
+ #  modification, are permitted provided that the following conditions are met:
+ #
+ #  Redistributions of source code must retain the above copyright notice, this
+ #  list of conditions and the following disclaimer. Redistributions in binary
+ #  form must reproduce the above copyright notice, this list of conditions and
+ #  the following disclaimer in the documentation and/or ther materials
+ #  provided with the distribution.
+ #
+ # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ # FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ # COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
+ # INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ # (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ # HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ # STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
+ # OF THE POSSIBILITY OF SUCH DAMAGE.
+ #  --------------------------------------------------------------------------
+
+4. Burmese Word Break Dictionary Data (burmesedict.txt)
+
+ #  Copyright (c) 2014 International Business Machines Corporation
+ #  and others. All Rights Reserved.
+ #
+ #  This list is part of a project hosted at:
+ #    github.com/kanyawtech/myanmar-karen-word-lists
+ #
+ #  --------------------------------------------------------------------------
+ #  Copyright (c) 2013, LeRoy Benjamin Sharon
+ #  All rights reserved.
+ #
+ #  Redistribution and use in source and binary forms, with or without
+ #  modification, are permitted provided that the following conditions
+ #  are met: Redistributions of source code must retain the above
+ #  copyright notice, this list of conditions and the following
+ #  disclaimer.  Redistributions in binary form must reproduce the
+ #  above copyright notice, this list of conditions and the following
+ #  disclaimer in the documentation and/or other materials provided
+ #  with the distribution.
+ #
+ #    Neither the name Myanmar Karen Word Lists, nor the names of its
+ #    contributors may be used to endorse or promote products derived
+ #    from this software without specific prior written permission.
+ #
+ #  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ #  CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ #  INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ #  MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ #  DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
+ #  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ #  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
+ #  TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ #  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ #  ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
+ #  TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
+ #  THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ #  SUCH DAMAGE.
+ #  --------------------------------------------------------------------------
+
+5. Time Zone Database
+
+  ICU uses the public domain data and code derived from Time Zone
+Database for its time zone support. The ownership of the TZ database
+is explained in BCP 175: Procedure for Maintaining the Time Zone
+Database section 7.
+
+ # 7.  Database Ownership
+ #
+ #    The TZ database itself is not an IETF Contribution or an IETF
+ #    document.  Rather it is a pre-existing and regularly updated work
+ #    that is in the public domain, and is intended to remain in the
+ #    public domain.  Therefore, BCPs 78 [RFC5378] and 79 [RFC3979] do
+ #    not apply to the TZ Database or contributions that individuals make
+ #    to it.  Should any claims be made and substantiated against the TZ
+ #    Database, the organization that is providing the IANA
+ #    Considerations defined in this RFC, under the memorandum of
+ #    understanding with the IETF, currently ICANN, may act in accordance
+ #    with all competent court orders.  No ownership claims will be made
+ #    by ICANN or the IETF Trust on the database or the code.  Any person
+ #    making a contribution to the database or code waives all rights to
+ #    future claims in that contribution or in the TZ Database.
+
+6. Google double-conversion
+
+Copyright 2006-2011, the V8 project authors. All rights reserved.
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above
+      copyright notice, this list of conditions and the following
+      disclaimer in the documentation and/or other materials provided
+      with the distribution.
+    * Neither the name of Google Inc. nor the names of its
+      contributors may be used to endorse or promote products derived
+      from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+    </pre>
+
+    <hr>
+
     <h3>Kotlin & KotlinX libraries</h3>
     <span>Copyright 2000-2020 JetBrains s.r.o. and Kotlin Programming Language contributors.</span>
     <pre>
@@ -711,6 +1127,176 @@ SOFTWARE.
 
     <hr>
 
+    <h3>Nuspell</h3>
+    <span>Copyright (c) 2021 Nuspell</span>
+    <pre>
+                   GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. &lt;http://fsf.org/&gt;
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+
+  This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+  0. Additional Definitions.
+
+  As used herein, "this License" refers to version 3 of the GNU Lesser
+General Public License, and the "GNU GPL" refers to version 3 of the GNU
+General Public License.
+
+  "The Library" refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+  An "Application" is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+  A "Combined Work" is a work produced by combining or linking an
+Application with the Library.  The particular version of the Library
+with which the Combined Work was made is also called the "Linked
+Version".
+
+  The "Minimal Corresponding Source" for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+  The "Corresponding Application Code" for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+  1. Exception to Section 3 of the GNU GPL.
+
+  You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+  2. Conveying Modified Versions.
+
+  If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+   a) under this License, provided that you make a good faith effort to
+   ensure that, in the event an Application does not supply the
+   function or data, the facility still operates, and performs
+   whatever part of its purpose remains meaningful, or
+
+   b) under the GNU GPL, with none of the additional permissions of
+   this License applicable to that copy.
+
+  3. Object Code Incorporating Material from Library Header Files.
+
+  The object code form of an Application may incorporate material from
+a header file that is part of the Library.  You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+   a) Give prominent notice with each copy of the object code that the
+   Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the object code with a copy of the GNU GPL and this license
+   document.
+
+  4. Combined Works.
+
+  You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+   a) Give prominent notice with each copy of the Combined Work that
+   the Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the Combined Work with a copy of the GNU GPL and this license
+   document.
+
+   c) For a Combined Work that displays copyright notices during
+   execution, include the copyright notice for the Library among
+   these notices, as well as a reference directing the user to the
+   copies of the GNU GPL and this license document.
+
+   d) Do one of the following:
+
+       0) Convey the Minimal Corresponding Source under the terms of this
+       License, and the Corresponding Application Code in a form
+       suitable for, and under terms that permit, the user to
+       recombine or relink the Application with a modified version of
+       the Linked Version to produce a modified Combined Work, in the
+       manner specified by section 6 of the GNU GPL for conveying
+       Corresponding Source.
+
+       1) Use a suitable shared library mechanism for linking with the
+       Library.  A suitable mechanism is one that (a) uses at run time
+       a copy of the Library already present on the user's computer
+       system, and (b) will operate properly with a modified version
+       of the Library that is interface-compatible with the Linked
+       Version.
+
+   e) Provide Installation Information, but only if you would otherwise
+   be required to provide such information under section 6 of the
+   GNU GPL, and only to the extent that such information is
+   necessary to install and execute a modified version of the
+   Combined Work produced by recombining or relinking the
+   Application with a modified version of the Linked Version. (If
+   you use option 4d0, the Installation Information must accompany
+   the Minimal Corresponding Source and Corresponding Application
+   Code. If you use option 4d1, you must provide the Installation
+   Information in the manner specified by section 6 of the GNU GPL
+   for conveying Corresponding Source.)
+
+  5. Combined Libraries.
+
+  You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+   a) Accompany the combined library with a copy of the same work based
+   on the Library, uncombined with any other library facilities,
+   conveyed under the terms of this License.
+
+   b) Give prominent notice with the combined library that part of it
+   is a work based on the Library, and explaining where to find the
+   accompanying uncombined form of the same work.
+
+  6. Revised Versions of the GNU Lesser General Public License.
+
+  The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+  Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License "or any later version"
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+  If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.
+    </pre>
+
     <h3>Timber</h3>
     <span>Copyright 2013 Jake Wharton</span>
     <pre>
@@ -764,384 +1350,6 @@ shall not be used in advertising or otherwise to promote the sale,
 use or other dealings in these Data Files or Software without prior
 written authorization of the copyright holder.
     </pre>
-
-    <hr>
-
-    <h3>Dictionary Source 1: wordfreq data</h3>
-    <span>Copyright (c) 2015 Luminoso Technologies, Inc.</span>
-    <pre>
-Creative Commons Attribution-ShareAlike 4.0 International Public
-License
-
-By exercising the Licensed Rights (defined below), You accept and agree
-to be bound by the terms and conditions of this Creative Commons
-Attribution-ShareAlike 4.0 International Public License ("Public
-License"). To the extent this Public License may be interpreted as a
-contract, You are granted the Licensed Rights in consideration of Your
-acceptance of these terms and conditions, and the Licensor grants You
-such rights in consideration of benefits the Licensor receives from
-making the Licensed Material available under these terms and
-conditions.
-
-
-Section 1 -- Definitions.
-
-  a. Adapted Material means material subject to Copyright and Similar
-     Rights that is derived from or based upon the Licensed Material
-     and in which the Licensed Material is translated, altered,
-     arranged, transformed, or otherwise modified in a manner requiring
-     permission under the Copyright and Similar Rights held by the
-     Licensor. For purposes of this Public License, where the Licensed
-     Material is a musical work, performance, or sound recording,
-     Adapted Material is always produced where the Licensed Material is
-     synched in timed relation with a moving image.
-
-  b. Adapter's License means the license You apply to Your Copyright
-     and Similar Rights in Your contributions to Adapted Material in
-     accordance with the terms and conditions of this Public License.
-
-  c. BY-SA Compatible License means a license listed at
-     creativecommons.org/compatiblelicenses, approved by Creative
-     Commons as essentially the equivalent of this Public License.
-
-  d. Copyright and Similar Rights means copyright and/or similar rights
-     closely related to copyright including, without limitation,
-     performance, broadcast, sound recording, and Sui Generis Database
-     Rights, without regard to how the rights are labeled or
-     categorized. For purposes of this Public License, the rights
-     specified in Section 2(b)(1)-(2) are not Copyright and Similar
-     Rights.
-
-  e. Effective Technological Measures means those measures that, in the
-     absence of proper authority, may not be circumvented under laws
-     fulfilling obligations under Article 11 of the WIPO Copyright
-     Treaty adopted on December 20, 1996, and/or similar international
-     agreements.
-
-  f. Exceptions and Limitations means fair use, fair dealing, and/or
-     any other exception or limitation to Copyright and Similar Rights
-     that applies to Your use of the Licensed Material.
-
-  g. License Elements means the license attributes listed in the name
-     of a Creative Commons Public License. The License Elements of this
-     Public License are Attribution and ShareAlike.
-
-  h. Licensed Material means the artistic or literary work, database,
-     or other material to which the Licensor applied this Public
-     License.
-
-  i. Licensed Rights means the rights granted to You subject to the
-     terms and conditions of this Public License, which are limited to
-     all Copyright and Similar Rights that apply to Your use of the
-     Licensed Material and that the Licensor has authority to license.
-
-  j. Licensor means the individual(s) or entity(ies) granting rights
-     under this Public License.
-
-  k. Share means to provide material to the public by any means or
-     process that requires permission under the Licensed Rights, such
-     as reproduction, public display, public performance, distribution,
-     dissemination, communication, or importation, and to make material
-     available to the public including in ways that members of the
-     public may access the material from a place and at a time
-     individually chosen by them.
-
-  l. Sui Generis Database Rights means rights other than copyright
-     resulting from Directive 96/9/EC of the European Parliament and of
-     the Council of 11 March 1996 on the legal protection of databases,
-     as amended and/or succeeded, as well as other essentially
-     equivalent rights anywhere in the world.
-
-  m. You means the individual or entity exercising the Licensed Rights
-     under this Public License. Your has a corresponding meaning.
-
-
-Section 2 -- Scope.
-
-  a. License grant.
-
-       1. Subject to the terms and conditions of this Public License,
-          the Licensor hereby grants You a worldwide, royalty-free,
-          non-sublicensable, non-exclusive, irrevocable license to
-          exercise the Licensed Rights in the Licensed Material to:
-
-            a. reproduce and Share the Licensed Material, in whole or
-               in part; and
-
-            b. produce, reproduce, and Share Adapted Material.
-
-       2. Exceptions and Limitations. For the avoidance of doubt, where
-          Exceptions and Limitations apply to Your use, this Public
-          License does not apply, and You do not need to comply with
-          its terms and conditions.
-
-       3. Term. The term of this Public License is specified in Section
-          6(a).
-
-       4. Media and formats; technical modifications allowed. The
-          Licensor authorizes You to exercise the Licensed Rights in
-          all media and formats whether now known or hereafter created,
-          and to make technical modifications necessary to do so. The
-          Licensor waives and/or agrees not to assert any right or
-          authority to forbid You from making technical modifications
-          necessary to exercise the Licensed Rights, including
-          technical modifications necessary to circumvent Effective
-          Technological Measures. For purposes of this Public License,
-          simply making modifications authorized by this Section 2(a)
-          (4) never produces Adapted Material.
-
-       5. Downstream recipients.
-
-            a. Offer from the Licensor -- Licensed Material. Every
-               recipient of the Licensed Material automatically
-               receives an offer from the Licensor to exercise the
-               Licensed Rights under the terms and conditions of this
-               Public License.
-
-            b. Additional offer from the Licensor -- Adapted Material.
-               Every recipient of Adapted Material from You
-               automatically receives an offer from the Licensor to
-               exercise the Licensed Rights in the Adapted Material
-               under the conditions of the Adapter's License You apply.
-
-            c. No downstream restrictions. You may not offer or impose
-               any additional or different terms or conditions on, or
-               apply any Effective Technological Measures to, the
-               Licensed Material if doing so restricts exercise of the
-               Licensed Rights by any recipient of the Licensed
-               Material.
-
-       6. No endorsement. Nothing in this Public License constitutes or
-          may be construed as permission to assert or imply that You
-          are, or that Your use of the Licensed Material is, connected
-          with, or sponsored, endorsed, or granted official status by,
-          the Licensor or others designated to receive attribution as
-          provided in Section 3(a)(1)(A)(i).
-
-  b. Other rights.
-
-       1. Moral rights, such as the right of integrity, are not
-          licensed under this Public License, nor are publicity,
-          privacy, and/or other similar personality rights; however, to
-          the extent possible, the Licensor waives and/or agrees not to
-          assert any such rights held by the Licensor to the limited
-          extent necessary to allow You to exercise the Licensed
-          Rights, but not otherwise.
-
-       2. Patent and trademark rights are not licensed under this
-          Public License.
-
-       3. To the extent possible, the Licensor waives any right to
-          collect royalties from You for the exercise of the Licensed
-          Rights, whether directly or through a collecting society
-          under any voluntary or waivable statutory or compulsory
-          licensing scheme. In all other cases the Licensor expressly
-          reserves any right to collect such royalties.
-
-
-Section 3 -- License Conditions.
-
-Your exercise of the Licensed Rights is expressly made subject to the
-following conditions.
-
-  a. Attribution.
-
-       1. If You Share the Licensed Material (including in modified
-          form), You must:
-
-            a. retain the following if it is supplied by the Licensor
-               with the Licensed Material:
-
-                 i. identification of the creator(s) of the Licensed
-                    Material and any others designated to receive
-                    attribution, in any reasonable manner requested by
-                    the Licensor (including by pseudonym if
-                    designated);
-
-                ii. a copyright notice;
-
-               iii. a notice that refers to this Public License;
-
-                iv. a notice that refers to the disclaimer of
-                    warranties;
-
-                 v. a URI or hyperlink to the Licensed Material to the
-                    extent reasonably practicable;
-
-            b. indicate if You modified the Licensed Material and
-               retain an indication of any previous modifications; and
-
-            c. indicate the Licensed Material is licensed under this
-               Public License, and include the text of, or the URI or
-               hyperlink to, this Public License.
-
-       2. You may satisfy the conditions in Section 3(a)(1) in any
-          reasonable manner based on the medium, means, and context in
-          which You Share the Licensed Material. For example, it may be
-          reasonable to satisfy the conditions by providing a URI or
-          hyperlink to a resource that includes the required
-          information.
-
-       3. If requested by the Licensor, You must remove any of the
-          information required by Section 3(a)(1)(A) to the extent
-          reasonably practicable.
-
-  b. ShareAlike.
-
-     In addition to the conditions in Section 3(a), if You Share
-     Adapted Material You produce, the following conditions also apply.
-
-       1. The Adapter's License You apply must be a Creative Commons
-          license with the same License Elements, this version or
-          later, or a BY-SA Compatible License.
-
-       2. You must include the text of, or the URI or hyperlink to, the
-          Adapter's License You apply. You may satisfy this condition
-          in any reasonable manner based on the medium, means, and
-          context in which You Share Adapted Material.
-
-       3. You may not offer or impose any additional or different terms
-          or conditions on, or apply any Effective Technological
-          Measures to, Adapted Material that restrict exercise of the
-          rights granted under the Adapter's License You apply.
-
-
-Section 4 -- Sui Generis Database Rights.
-
-Where the Licensed Rights include Sui Generis Database Rights that
-apply to Your use of the Licensed Material:
-
-  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
-     to extract, reuse, reproduce, and Share all or a substantial
-     portion of the contents of the database;
-
-  b. if You include all or a substantial portion of the database
-     contents in a database in which You have Sui Generis Database
-     Rights, then the database in which You have Sui Generis Database
-     Rights (but not its individual contents) is Adapted Material,
-
-     including for purposes of Section 3(b); and
-  c. You must comply with the conditions in Section 3(a) if You Share
-     all or a substantial portion of the contents of the database.
-
-For the avoidance of doubt, this Section 4 supplements and does not
-replace Your obligations under this Public License where the Licensed
-Rights include other Copyright and Similar Rights.
-
-
-Section 5 -- Disclaimer of Warranties and Limitation of Liability.
-
-  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
-     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
-     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
-     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
-     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
-     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
-     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
-     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
-     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
-     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
-
-  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
-     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
-     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
-     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
-     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
-     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
-     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
-     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
-     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
-
-  c. The disclaimer of warranties and limitation of liability provided
-     above shall be interpreted in a manner that, to the extent
-     possible, most closely approximates an absolute disclaimer and
-     waiver of all liability.
-
-
-Section 6 -- Term and Termination.
-
-  a. This Public License applies for the term of the Copyright and
-     Similar Rights licensed here. However, if You fail to comply with
-     this Public License, then Your rights under this Public License
-     terminate automatically.
-
-  b. Where Your right to use the Licensed Material has terminated under
-     Section 6(a), it reinstates:
-
-       1. automatically as of the date the violation is cured, provided
-          it is cured within 30 days of Your discovery of the
-          violation; or
-
-       2. upon express reinstatement by the Licensor.
-
-     For the avoidance of doubt, this Section 6(b) does not affect any
-     right the Licensor may have to seek remedies for Your violations
-     of this Public License.
-
-  c. For the avoidance of doubt, the Licensor may also offer the
-     Licensed Material under separate terms or conditions or stop
-     distributing the Licensed Material at any time; however, doing so
-     will not terminate this Public License.
-
-  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
-     License.
-
-
-Section 7 -- Other Terms and Conditions.
-
-  a. The Licensor shall not be bound by any additional or different
-     terms or conditions communicated by You unless expressly agreed.
-
-  b. Any arrangements, understandings, or agreements regarding the
-     Licensed Material not stated herein are separate from and
-     independent of the terms and conditions of this Public License.
-
-
-Section 8 -- Interpretation.
-
-  a. For the avoidance of doubt, this Public License does not, and
-     shall not be interpreted to, reduce, limit, restrict, or impose
-     conditions on any use of the Licensed Material that could lawfully
-     be made without permission under this Public License.
-
-  b. To the extent possible, if any provision of this Public License is
-     deemed unenforceable, it shall be automatically reformed to the
-     minimum extent necessary to make it enforceable. If the provision
-     cannot be reformed, it shall be severed from this Public License
-     without affecting the enforceability of the remaining terms and
-     conditions.
-
-  c. No term or condition of this Public License will be waived and no
-     failure to comply consented to unless expressly agreed to by the
-     Licensor.
-
-  d. Nothing in this Public License constitutes or may be interpreted
-     as a limitation upon, or waiver of, any privileges and immunities
-     that apply to the Licensor or You, including from the legal
-     processes of any jurisdiction or authority.
-
-
-=======================================================================
-
-Creative Commons is not a party to its public
-licenses. Notwithstanding, Creative Commons may elect to apply one of
-its public licenses to material it publishes and in those instances
-will be considered the “Licensor.” The text of the Creative Commons
-public licenses is dedicated to the public domain under the CC0 Public
-Domain Dedication. Except for the limited purpose of indicating that
-material is shared under a Creative Commons public license or as
-otherwise permitted by the Creative Commons policies published at
-creativecommons.org/policies, Creative Commons does not authorize the
-use of the trademark "Creative Commons" or any other trademark or logo
-of Creative Commons without its prior written consent including,
-without limitation, in connection with any unauthorized modifications
-to any of its public licenses or any other arrangements,
-understandings, or agreements concerning use of licensed material. For
-the avoidance of doubt, this paragraph does not form part of the
-public licenses.
-
-Creative Commons may be contacted at creativecommons.org.
-    </pre>
 </body>
 
 </html>

app/src/main/cpp/CMakeLists.txt

@@ -7,32 +7,42 @@ project("florisboard")
 
 set(CMAKE_CXX_STANDARD 17)
 
+include_directories(.)
+
+### ICU4C ###
+include_directories(../icu4c/prebuilt/include)
+set(JNI_LIBS ${CMAKE_SOURCE_DIR}/../icu4c/prebuilt/jniLibs/${ANDROID_ABI})
+add_library(ICU::data STATIC IMPORTED)
+set_property(TARGET ICU::data PROPERTY IMPORTED_LOCATION "${JNI_LIBS}/libicudata.a")
+add_library(ICU::uc STATIC IMPORTED)
+set_property(TARGET ICU::uc PROPERTY IMPORTED_LOCATION "${JNI_LIBS}/libicuuc.a")
+
+### FlorisBoard ###
+add_subdirectory(nuspell)
+add_subdirectory(utils)
 add_subdirectory(ime/nlp)
+add_subdirectory(ime/spelling)
 
 add_library(
-        # Name
         florisboard-native
-
-        # Type
         SHARED
-
-        # Sources
+        dev_patrickgold_florisboard_FlorisApplication.cpp
         dev_patrickgold_florisboard_ime_nlp_SuggestionList.cpp
+        dev_patrickgold_florisboard_ime_spelling_SpellingDict.cpp
 )
 
-find_library(
-        # Save to var
-        log-lib
-
-        # Original name
-        log
-)
-
+target_compile_options(florisboard-native PRIVATE -ffunction-sections -fdata-sections -fexceptions)
 target_link_libraries(
         # Destination
         florisboard-native
 
         # Sources
-        ${log-lib}
+        android
+        log
+        ICU::uc
+        ICU::data
+        Nuspell::nuspell
+        utils
         ime-nlp
+        ime-spelling
 )

app/src/main/cpp/dev_patrickgold_florisboard_FlorisApplication.cpp

@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <fstream>
+#include <vector>
+#include <jni.h>
+#include <unicode/udata.h>
+#include "utils/jni_utils.h"
+
+#pragma ide diagnostic ignored "UnusedLocalVariable"
+
+extern "C"
+JNIEXPORT jint JNICALL
+Java_dev_patrickgold_florisboard_FlorisApplication_00024Companion_nativeInitICUData(
+        JNIEnv *env,
+        jobject thiz,
+        jobject path) {
+    auto path_str = utils::j2std_string(env, path);
+    std::ifstream in_file(path_str, std::ios::in | std::ios::binary);
+    if (!in_file) {
+        return U_FILE_ACCESS_ERROR;
+    }
+    in_file.seekg(0, std::ios::end);
+    size_t size = in_file.tellg();
+    if (size <= 0) {
+        return U_FILE_ACCESS_ERROR;
+    }
+    in_file.seekg(0, std::ios::beg);
+    char *icu_data = new char[size + 1];
+    in_file.read(icu_data, size);
+    if (!in_file) {
+        in_file.close();
+        return U_FILE_ACCESS_ERROR;
+    }
+    icu_data[size] = 0;
+    in_file.close();
+    UErrorCode status = U_ZERO_ERROR;
+    udata_setCommonData(reinterpret_cast<void *>(icu_data), &status);
+    return status;
+}

app/src/main/cpp/dev_patrickgold_florisboard_ime_spelling_SpellingDict.cpp

@@ -0,0 +1,90 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <jni.h>
+#include <algorithm>
+#include "ime/spelling/spellingdict.h"
+#include "utils/jni_utils.h"
+
+#pragma ide diagnostic ignored "UnusedLocalVariable"
+
+using namespace ime::spellcheck;
+
+extern "C"
+JNIEXPORT jlong JNICALL
+Java_dev_patrickgold_florisboard_ime_spelling_SpellingDict_00024Companion_nativeInitialize(
+        JNIEnv *env,
+        jobject thiz,
+        jobject base_path) {
+    auto strBasePath = utils::j2std_string(env, base_path);
+
+    auto *spellingDict = SpellingDict::load(strBasePath);
+
+    if (spellingDict == nullptr) {
+        return 0L;
+    } else {
+        return reinterpret_cast<jlong>(spellingDict);
+    }
+}
+
+extern "C"
+JNIEXPORT void JNICALL
+Java_dev_patrickgold_florisboard_ime_spelling_SpellingDict_00024Companion_nativeDispose(
+        JNIEnv *env,
+        jobject thiz,
+        jlong native_ptr) {
+    auto spellingDict = reinterpret_cast<SpellingDict *>(native_ptr);
+
+    delete spellingDict;
+}
+
+extern "C"
+JNIEXPORT jboolean JNICALL
+Java_dev_patrickgold_florisboard_ime_spelling_SpellingDict_00024Companion_nativeSpell(
+        JNIEnv *env,
+        jobject thiz,
+        jlong native_ptr,
+        jobject word) {
+    auto strWord = utils::j2std_string(env, word);
+
+    auto spellingDict = reinterpret_cast<SpellingDict *>(native_ptr);
+    auto result = spellingDict->spell(strWord);
+
+    return result;
+}
+
+extern "C"
+JNIEXPORT jobjectArray JNICALL
+Java_dev_patrickgold_florisboard_ime_spelling_SpellingDict_00024Companion_nativeSuggest(
+        JNIEnv *env,
+        jobject thiz,
+        jlong native_ptr,
+        jobject word,
+        jint limit) {
+    auto strWord = utils::j2std_string(env, word);
+
+    auto spellingDict = reinterpret_cast<SpellingDict *>(native_ptr);
+    auto result = spellingDict->suggest(strWord);
+    auto retSize = std::min(result.size(), (size_t)std::max(0, limit));
+
+    jclass jByteArrayClass = env->FindClass("java/nio/ByteBuffer");
+    jobjectArray jSuggestions = env->NewObjectArray(retSize, jByteArrayClass, nullptr);
+    for (int n = 0; n < retSize; n++) {
+        env->SetObjectArrayElement(jSuggestions, n, utils::std2j_string(env, result[n]));
+    }
+
+    return jSuggestions;
+}

app/src/main/cpp/ime/spelling/CMakeLists.txt

@@ -0,0 +1,10 @@
+add_library(
+        # Name
+        ime-spelling
+
+        # Headers
+        spellingdict.h
+
+        # Sources
+        spellingdict.cpp
+)

app/src/main/cpp/ime/spelling/spellingdict.cpp

@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "spellingdict.h"
+#include "utils/log.h"
+
+using namespace ime::spellcheck;
+
+SpellingDict::SpellingDict(const nuspell::Dictionary& dict) : dictionary(std::make_unique<nuspell::Dictionary>(dict))
+{ }
+
+SpellingDict::~SpellingDict() = default;
+
+SpellingDict* SpellingDict::load(const std::string &basePath) {
+    utils::start_stdout_stderr_logger("spell-floris");
+    try {
+        auto temp = nuspell::Dictionary::load_from_path(basePath);
+        auto spellingDict = new SpellingDict(temp);
+        return spellingDict;
+    } catch (const nuspell::Dictionary_Loading_Error& e) {
+        utils::log_error("SpellingDict.load()", e.what());
+        return nullptr;
+    } catch (...) {
+        utils::log_error("SpellingDict.load()", "An unknown error occurred!");
+        return nullptr;
+    }
+}
+
+bool SpellingDict::spell(const std::string& word) {
+    bool result = dictionary->spell(word);
+    return result;
+}
+
+std::vector<std::string> SpellingDict::suggest(const std::string &word) {
+    auto result = std::vector<std::string>();
+    dictionary->suggest(word, result);
+    return result;
+}

app/src/main/cpp/ime/spelling/spellingdict.h

@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef FLORISBOARD_SPELLINGDICT_H
+#define FLORISBOARD_SPELLINGDICT_H
+
+#include "nuspell/dictionary.hxx"
+#include <string>
+#include <vector>
+
+namespace ime::spellcheck {
+
+class SpellingDict {
+public:
+    SpellingDict(const nuspell::Dictionary& dict);
+    ~SpellingDict();
+
+    static SpellingDict* load(const std::string& basePath);
+
+    bool spell(const std::string& word);
+    std::vector<std::string> suggest(const std::string& word);
+
+private:
+    std::unique_ptr<nuspell::Dictionary> dictionary;
+};
+
+} // namespace ime::spellcheck
+
+#endif // FLORISBOARD_SPELLINGDICT_H

app/src/main/cpp/nuspell/CMakeLists.txt

@@ -0,0 +1,10 @@
+add_library(nuspell
+aff_data.cxx     aff_data.hxx
+checker.cxx      checker.hxx
+suggester.cxx    suggester.hxx
+dictionary.cxx   dictionary.hxx
+                 unicode.hxx
+utils.cxx        utils.hxx
+                 structures.hxx)
+
+add_library(Nuspell::nuspell ALIAS nuspell)

app/src/main/cpp/nuspell/LICENSE.txt

@@ -0,0 +1,165 @@
+                   GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+
+  This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+  0. Additional Definitions.
+
+  As used herein, "this License" refers to version 3 of the GNU Lesser
+General Public License, and the "GNU GPL" refers to version 3 of the GNU
+General Public License.
+
+  "The Library" refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+  An "Application" is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+  A "Combined Work" is a work produced by combining or linking an
+Application with the Library.  The particular version of the Library
+with which the Combined Work was made is also called the "Linked
+Version".
+
+  The "Minimal Corresponding Source" for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+  The "Corresponding Application Code" for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+  1. Exception to Section 3 of the GNU GPL.
+
+  You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+  2. Conveying Modified Versions.
+
+  If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+   a) under this License, provided that you make a good faith effort to
+   ensure that, in the event an Application does not supply the
+   function or data, the facility still operates, and performs
+   whatever part of its purpose remains meaningful, or
+
+   b) under the GNU GPL, with none of the additional permissions of
+   this License applicable to that copy.
+
+  3. Object Code Incorporating Material from Library Header Files.
+
+  The object code form of an Application may incorporate material from
+a header file that is part of the Library.  You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+   a) Give prominent notice with each copy of the object code that the
+   Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the object code with a copy of the GNU GPL and this license
+   document.
+
+  4. Combined Works.
+
+  You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+   a) Give prominent notice with each copy of the Combined Work that
+   the Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the Combined Work with a copy of the GNU GPL and this license
+   document.
+
+   c) For a Combined Work that displays copyright notices during
+   execution, include the copyright notice for the Library among
+   these notices, as well as a reference directing the user to the
+   copies of the GNU GPL and this license document.
+
+   d) Do one of the following:
+
+       0) Convey the Minimal Corresponding Source under the terms of this
+       License, and the Corresponding Application Code in a form
+       suitable for, and under terms that permit, the user to
+       recombine or relink the Application with a modified version of
+       the Linked Version to produce a modified Combined Work, in the
+       manner specified by section 6 of the GNU GPL for conveying
+       Corresponding Source.
+
+       1) Use a suitable shared library mechanism for linking with the
+       Library.  A suitable mechanism is one that (a) uses at run time
+       a copy of the Library already present on the user's computer
+       system, and (b) will operate properly with a modified version
+       of the Library that is interface-compatible with the Linked
+       Version.
+
+   e) Provide Installation Information, but only if you would otherwise
+   be required to provide such information under section 6 of the
+   GNU GPL, and only to the extent that such information is
+   necessary to install and execute a modified version of the
+   Combined Work produced by recombining or relinking the
+   Application with a modified version of the Linked Version. (If
+   you use option 4d0, the Installation Information must accompany
+   the Minimal Corresponding Source and Corresponding Application
+   Code. If you use option 4d1, you must provide the Installation
+   Information in the manner specified by section 6 of the GNU GPL
+   for conveying Corresponding Source.)
+
+  5. Combined Libraries.
+
+  You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+   a) Accompany the combined library with a copy of the same work based
+   on the Library, uncombined with any other library facilities,
+   conveyed under the terms of this License.
+
+   b) Give prominent notice with the combined library that part of it
+   is a work based on the Library, and explaining where to find the
+   accompanying uncombined form of the same work.
+
+  6. Revised Versions of the GNU Lesser General Public License.
+
+  The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+  Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License "or any later version"
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+  If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.

app/src/main/cpp/nuspell/aff_data.hxx

@@ -0,0 +1,173 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef NUSPELL_AFF_DATA_HXX
+#define NUSPELL_AFF_DATA_HXX
+
+#include "nuspell_export.h"
+#include "structures.hxx"
+
+#include <iosfwd>
+#include <unicode/locid.h>
+
+namespace nuspell {
+inline namespace v5 {
+
+class Encoding {
+	std::string name;
+
+	NUSPELL_EXPORT auto normalize_name() -> void;
+
+      public:
+	enum Enc_Type { SINGLEBYTE = false, UTF8 = true };
+
+	Encoding() = default;
+	explicit Encoding(const std::string& e) : name(e) { normalize_name(); }
+	explicit Encoding(std::string&& e) : name(move(e)) { normalize_name(); }
+	explicit Encoding(const char* e) : name(e) { normalize_name(); }
+	auto& operator=(const std::string& e)
+	{
+		name = e;
+		normalize_name();
+		return *this;
+	}
+	auto& operator=(std::string&& e)
+	{
+		name = move(e);
+		normalize_name();
+		return *this;
+	}
+	auto& operator=(const char* e)
+	{
+		name = e;
+		normalize_name();
+		return *this;
+	}
+	auto empty() const { return name.empty(); }
+	auto& value() const { return name; }
+	auto is_utf8() const { return name == "UTF-8"; }
+	auto value_or_default() const -> std::string
+	{
+		if (name.empty())
+			return "ISO8859-1";
+		else
+			return name;
+	}
+	operator Enc_Type() const { return is_utf8() ? UTF8 : SINGLEBYTE; }
+};
+
+enum class Flag_Type { SINGLE_CHAR, DOUBLE_CHAR, NUMBER, UTF8 };
+
+/**
+ * @internal
+ * @brief Map between words and word_flags.
+ *
+ * Flags are stored as part of the container. Maybe for the future flags should
+ * be stored elsewhere (flag aliases) and this should store pointers.
+ *
+ * Does not store morphological data as is low priority feature and is out of
+ * scope.
+ */
+using Word_List = Hash_Multimap<std::string, Flag_Set>;
+
+struct Aff_Data {
+	static constexpr auto HIDDEN_HOMONYM_FLAG = char16_t(-1);
+	static constexpr auto MAX_SUGGESTIONS = size_t(16);
+
+	// spell checking options
+	Word_List words;
+	Prefix_Table prefixes;
+	Suffix_Table suffixes;
+
+	bool complex_prefixes;
+	bool fullstrip;
+	bool checksharps;
+	bool forbid_warn;
+	char16_t compound_onlyin_flag;
+	char16_t circumfix_flag;
+	char16_t forbiddenword_flag;
+	char16_t keepcase_flag;
+	char16_t need_affix_flag;
+	char16_t warn_flag;
+
+	// compounding options
+	char16_t compound_flag;
+	char16_t compound_begin_flag;
+	char16_t compound_last_flag;
+	char16_t compound_middle_flag;
+	Compound_Rule_Table compound_rules;
+
+	// spell checking options
+	Break_Table break_table;
+	Substr_Replacer input_substr_replacer;
+	std::string ignored_chars;
+	icu::Locale icu_locale;
+	Substr_Replacer output_substr_replacer;
+
+	// suggestion options
+	Replacement_Table replacements;
+	std::vector<Similarity_Group> similarities;
+	std::string keyboard_closeness;
+	std::string try_chars;
+	// Phonetic_Table phonetic_table;
+
+	char16_t nosuggest_flag;
+	char16_t substandard_flag;
+	unsigned short max_compound_suggestions;
+	unsigned short max_ngram_suggestions;
+	unsigned short max_diff_factor;
+	bool only_max_diff;
+	bool no_split_suggestions;
+	bool suggest_with_dots;
+
+	// compounding options
+	unsigned short compound_min_length;
+	unsigned short compound_max_word_count;
+	char16_t compound_permit_flag;
+	char16_t compound_forbid_flag;
+	char16_t compound_root_flag;
+	char16_t compound_force_uppercase;
+	bool compound_more_suffixes;
+	bool compound_check_duplicate;
+	bool compound_check_rep;
+	bool compound_check_case;
+	bool compound_check_triple;
+	bool compound_simplified_triple;
+	bool compound_syllable_num;
+	unsigned short compound_syllable_max;
+	std::string compound_syllable_vowels;
+	std::vector<Compound_Pattern> compound_patterns;
+
+	// data members used only while parsing
+	Flag_Type flag_type;
+	Encoding encoding;
+	std::vector<Flag_Set> flag_aliases;
+	std::string wordchars; // deprecated?
+
+	auto parse_aff(std::istream& in) -> bool;
+	auto parse_dic(std::istream& in) -> bool;
+	auto parse_aff_dic(std::istream& aff, std::istream& dic)
+	{
+		if (parse_aff(aff))
+			return parse_dic(dic);
+		return false;
+	}
+};
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_AFF_DATA_HXX

app/src/main/cpp/nuspell/checker.hxx

@@ -0,0 +1,352 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef NUSPELL_CHECKER_HXX
+#define NUSPELL_CHECKER_HXX
+
+#include "aff_data.hxx"
+
+namespace nuspell {
+inline namespace v5 {
+
+enum Affixing_Mode {
+	FULL_WORD,
+	AT_COMPOUND_BEGIN,
+	AT_COMPOUND_END,
+	AT_COMPOUND_MIDDLE
+};
+
+struct Affixing_Result_Base {
+	Word_List::const_pointer root_word = {};
+
+	operator Word_List::const_pointer() const { return root_word; }
+	auto& operator*() const { return *root_word; }
+	auto operator->() const { return root_word; }
+};
+
+template <class T1 = void, class T2 = void>
+struct Affixing_Result : Affixing_Result_Base {
+	const T1* a = {};
+	const T2* b = {};
+
+	Affixing_Result() = default;
+	Affixing_Result(Word_List::const_reference r, const T1& a, const T2& b)
+	    : Affixing_Result_Base{&r}, a{&a}, b{&b}
+	{
+	}
+};
+template <class T1>
+struct Affixing_Result<T1, void> : Affixing_Result_Base {
+	const T1* a = {};
+
+	Affixing_Result() = default;
+	Affixing_Result(Word_List::const_reference r, const T1& a)
+	    : Affixing_Result_Base{&r}, a{&a}
+	{
+	}
+};
+
+template <>
+struct Affixing_Result<void, void> : Affixing_Result_Base {
+	Affixing_Result() = default;
+	Affixing_Result(Word_List::const_reference r) : Affixing_Result_Base{&r}
+	{
+	}
+};
+
+struct Compounding_Result {
+	Word_List::const_pointer word_entry = {};
+	unsigned char num_words_modifier = {};
+	signed char num_syllable_modifier = {};
+	bool affixed_and_modified = {}; /**< non-zero affix */
+	operator Word_List::const_pointer() const { return word_entry; }
+	auto& operator*() const { return *word_entry; }
+	auto operator->() const { return word_entry; }
+};
+
+struct Checker : public Aff_Data {
+	enum Forceucase : bool {
+		FORBID_BAD_FORCEUCASE = false,
+		ALLOW_BAD_FORCEUCASE = true
+	};
+
+	enum Hidden_Homonym : bool {
+		ACCEPT_HIDDEN_HOMONYM = false,
+		SKIP_HIDDEN_HOMONYM = true
+	};
+	Checker()
+	    : Aff_Data() // we explicity do value init so content is zeroed
+	{
+	}
+	auto spell_priv(std::string& s) const -> bool;
+	auto spell_break(std::string& s, size_t depth = 0) const -> bool;
+	auto spell_casing(std::string& s) const -> const Flag_Set*;
+	auto spell_casing_upper(std::string& s) const -> const Flag_Set*;
+	auto spell_casing_title(std::string& s) const -> const Flag_Set*;
+	auto spell_sharps(std::string& base, size_t n_pos = 0, size_t n = 0,
+	                  size_t rep = 0) const -> const Flag_Set*;
+
+	auto check_word(std::string& s, Forceucase allow_bad_forceucase = {},
+	                Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> const Flag_Set*;
+	auto check_simple_word(std::string& word,
+	                       Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> const Flag_Set*;
+
+	template <Affixing_Mode m>
+	auto affix_NOT_valid(const Prefix& a) const;
+	template <Affixing_Mode m>
+	auto affix_NOT_valid(const Suffix& a) const;
+	template <Affixing_Mode m, class AffixT>
+	auto outer_affix_NOT_valid(const AffixT& a) const;
+	template <class AffixT>
+	auto is_circumfix(const AffixT& a) const;
+	template <Affixing_Mode m>
+	auto is_valid_inside_compound(const Flag_Set& flags) const;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_prefix_only(std::string& s,
+	                       Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Prefix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_suffix_only(std::string& s,
+	                       Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Suffix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto
+	strip_prefix_then_suffix(std::string& s,
+	                         Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Suffix, Prefix>;
+
+	template <Affixing_Mode m>
+	auto strip_pfx_then_sfx_2(const Prefix& pe, std::string& s,
+	                          Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<Suffix, Prefix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto
+	strip_suffix_then_prefix(std::string& s,
+	                         Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Prefix, Suffix>;
+
+	template <Affixing_Mode m>
+	auto strip_sfx_then_pfx_2(const Suffix& se, std::string& s,
+	                          Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<Prefix, Suffix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_prefix_then_suffix_commutative(
+	    std::string& word, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Suffix, Prefix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_pfx_then_sfx_comm_2(const Prefix& pe, std::string& word,
+	                               Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<Suffix, Prefix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto
+	strip_suffix_then_suffix(std::string& s,
+	                         Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Suffix, Suffix>;
+
+	template <Affixing_Mode m>
+	auto strip_sfx_then_sfx_2(const Suffix& se1, std::string& s,
+	                          Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<Suffix, Suffix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto
+	strip_prefix_then_prefix(std::string& s,
+	                         Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<Prefix, Prefix>;
+
+	template <Affixing_Mode m>
+	auto strip_pfx_then_pfx_2(const Prefix& pe1, std::string& s,
+	                          Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<Prefix, Prefix>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_prefix_then_2_suffixes(
+	    std::string& s, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_pfx_2_sfx_3(const Prefix& pe1, const Suffix& se1,
+	                       std::string& s,
+	                       Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_suffix_prefix_suffix(
+	    std::string& s, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_s_p_s_3(const Suffix& se1, const Prefix& pe1,
+	                   std::string& word,
+	                   Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_2_suffixes_then_prefix(
+	    std::string& s, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_2_sfx_pfx_3(const Suffix& se1, const Suffix& se2,
+	                       std::string& word,
+	                       Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_suffix_then_2_prefixes(
+	    std::string& s, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_sfx_2_pfx_3(const Suffix& se1, const Prefix& pe1,
+	                       std::string& s,
+	                       Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_prefix_suffix_prefix(
+	    std::string& word, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_p_s_p_3(const Prefix& pe1, const Suffix& se1,
+	                   std::string& word,
+	                   Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m = FULL_WORD>
+	auto strip_2_prefixes_then_suffix(
+	    std::string& word, Hidden_Homonym skip_hidden_homonym = {}) const
+	    -> Affixing_Result<>;
+
+	template <Affixing_Mode m>
+	auto strip_2_pfx_sfx_3(const Prefix& pe1, const Prefix& pe2,
+	                       std::string& word,
+	                       Hidden_Homonym skip_hidden_homonym) const
+	    -> Affixing_Result<>;
+
+	auto check_compound(std::string& word,
+	                    Forceucase allow_bad_forceucase) const
+	    -> Compounding_Result;
+
+	template <Affixing_Mode m = AT_COMPOUND_BEGIN>
+	auto check_compound(std::string& word, size_t start_pos,
+	                    size_t num_part, std::string& part,
+	                    Forceucase allow_bad_forceucase) const
+	    -> Compounding_Result;
+
+	template <Affixing_Mode m = AT_COMPOUND_BEGIN>
+	auto check_compound_classic(std::string& word, size_t start_pos,
+	                            size_t i, size_t num_part,
+	                            std::string& part,
+	                            Forceucase allow_bad_forceucase) const
+	    -> Compounding_Result;
+
+	template <Affixing_Mode m = AT_COMPOUND_BEGIN>
+	auto check_compound_with_pattern_replacements(
+	    std::string& word, size_t start_pos, size_t i, size_t num_part,
+	    std::string& part, Forceucase allow_bad_forceucase) const
+	    -> Compounding_Result;
+
+	template <Affixing_Mode m>
+	auto check_word_in_compound(std::string& s) const -> Compounding_Result;
+
+	auto calc_num_words_modifier(const Prefix& pfx) const -> unsigned char;
+
+	template <Affixing_Mode m>
+	auto calc_syllable_modifier(Word_List::const_reference we) const
+	    -> signed char;
+
+	template <Affixing_Mode m>
+	auto calc_syllable_modifier(Word_List::const_reference we,
+	                            const Suffix& sfx) const -> signed char;
+
+	auto count_syllables(std::string_view word) const -> size_t;
+
+	auto check_compound_with_rules(std::string& word,
+	                               std::vector<const Flag_Set*>& words_data,
+	                               size_t start_pos, std::string& part,
+	                               Forceucase allow_bad_forceucase) const
+
+	    -> Compounding_Result;
+	auto is_rep_similar(std::string& word) const -> bool;
+};
+
+template <Affixing_Mode m>
+auto Checker::affix_NOT_valid(const Prefix& e) const
+{
+	if (m == FULL_WORD && e.cont_flags.contains(compound_onlyin_flag))
+		return true;
+	if (m == AT_COMPOUND_END &&
+	    !e.cont_flags.contains(compound_permit_flag))
+		return true;
+	if (m != FULL_WORD && e.cont_flags.contains(compound_forbid_flag))
+		return true;
+	return false;
+}
+template <Affixing_Mode m>
+auto Checker::affix_NOT_valid(const Suffix& e) const
+{
+	if (m == FULL_WORD && e.cont_flags.contains(compound_onlyin_flag))
+		return true;
+	if (m == AT_COMPOUND_BEGIN &&
+	    !e.cont_flags.contains(compound_permit_flag))
+		return true;
+	if (m != FULL_WORD && e.cont_flags.contains(compound_forbid_flag))
+		return true;
+	return false;
+}
+template <Affixing_Mode m, class AffixT>
+auto Checker::outer_affix_NOT_valid(const AffixT& e) const
+{
+	if (affix_NOT_valid<m>(e))
+		return true;
+	if (e.cont_flags.contains(need_affix_flag))
+		return true;
+	return false;
+}
+template <class AffixT>
+auto Checker::is_circumfix(const AffixT& a) const
+{
+	return a.cont_flags.contains(circumfix_flag);
+}
+
+template <class AffixInner, class AffixOuter>
+auto cross_valid_inner_outer(const AffixInner& inner, const AffixOuter& outer)
+{
+	return inner.cont_flags.contains(outer.flag);
+}
+
+template <class Affix>
+auto cross_valid_inner_outer(const Flag_Set& word_flags, const Affix& afx)
+{
+	return word_flags.contains(afx.flag);
+}
+
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_CHECKER_HXX

app/src/main/cpp/nuspell/clang-format.sh

@@ -0,0 +1 @@
+clang-format -style=file -i *.[ch]xx

app/src/main/cpp/nuspell/dictionary.cxx

@@ -0,0 +1,115 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "dictionary.hxx"
+#include "utils.hxx"
+
+#include <fstream>
+#include <iostream>
+#include <stdexcept>
+
+using namespace std;
+
+namespace nuspell {
+inline namespace v5 {
+
+Dictionary::Dictionary(std::istream& aff, std::istream& dic)
+{
+	if (!parse_aff_dic(aff, dic))
+		throw Dictionary_Loading_Error("error parsing");
+}
+
+Dictionary::Dictionary() = default;
+
+/**
+ * @brief Create a dictionary from opened files as iostreams
+ *
+ * Prefer using load_from_path(). Use this if you have a specific use case,
+ * like when .aff and .dic are in-memory buffers istringstream.
+ *
+ * @param aff The iostream of the .aff file
+ * @param dic The iostream of the .dic file
+ * @return Dictionary object
+ * @throws Dictionary_Loading_Error on error
+ */
+auto Dictionary::load_from_aff_dic(std::istream& aff, std::istream& dic)
+    -> Dictionary
+{
+	return Dictionary(aff, dic);
+}
+
+/**
+ * @brief Create a dictionary from files
+ * @param file_path_without_extension path *without* extensions (without .dic or
+ * .aff)
+ * @return Dictionary object
+ * @throws Dictionary_Loading_Error on error
+ */
+auto Dictionary::load_from_path(const std::string& file_path_without_extension)
+    -> Dictionary
+{
+	auto path = file_path_without_extension;
+	path += ".aff";
+	std::ifstream aff_file(path);
+	if (aff_file.fail()) {
+		auto err = "Aff file " + path + " not found";
+		throw Dictionary_Loading_Error(err);
+	}
+	path.replace(path.size() - 3, 3, "dic");
+	std::ifstream dic_file(path);
+	if (dic_file.fail()) {
+		auto err = "Dic file " + path + " not found";
+		throw Dictionary_Loading_Error(err);
+	}
+	return load_from_aff_dic(aff_file, dic_file);
+}
+
+/**
+ * @brief Checks if a given word is correct
+ * @param word any word
+ * @return true if correct, false otherwise
+ */
+auto Dictionary::spell(std::string_view word) const -> bool
+{
+	auto ok_enc = validate_utf8(word);
+	if (unlikely(word.size() > 360))
+		return false;
+	if (unlikely(!ok_enc))
+		return false;
+	auto word_buf = string(word);
+	return spell_priv(word_buf);
+}
+
+/**
+ * @brief Suggests correct words for a given incorrect word
+ * @param[in] word incorrect word
+ * @param[out] out this object will be populated with the suggestions
+ */
+auto Dictionary::suggest(std::string_view word,
+                         std::vector<std::string>& out) const -> void
+{
+	out.clear();
+	auto ok_enc = validate_utf8(word);
+	if (unlikely(word.size() > 360))
+		return;
+	if (unlikely(!ok_enc))
+		return;
+	suggest_priv(word, out);
+}
+} // namespace v5
+} // namespace nuspell

app/src/main/cpp/nuspell/dictionary.hxx

@@ -0,0 +1,59 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/**
+ * @file
+ * @brief Dictionary spelling.
+ */
+
+#ifndef NUSPELL_DICTIONARY_HXX
+#define NUSPELL_DICTIONARY_HXX
+
+#include "suggester.hxx"
+
+namespace nuspell {
+inline namespace v5 {
+
+/**
+ * @brief The only important public exception
+ */
+class Dictionary_Loading_Error : public std::runtime_error {
+      public:
+	using std::runtime_error::runtime_error;
+};
+
+/**
+ * @brief The only important public class
+ */
+class NUSPELL_EXPORT Dictionary : private Suggester {
+	Dictionary(std::istream& aff, std::istream& dic);
+
+      public:
+	Dictionary();
+	auto static load_from_aff_dic(std::istream& aff, std::istream& dic)
+	    -> Dictionary;
+	auto static load_from_path(
+	    const std::string& file_path_without_extension) -> Dictionary;
+	auto spell(std::string_view word) const -> bool;
+	auto suggest(std::string_view word, std::vector<std::string>& out) const
+	    -> void;
+};
+
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_DICTIONARY_HXX

app/src/main/cpp/nuspell/nuspell_export.h

@@ -0,0 +1,18 @@
+#ifndef NUSPELL_EXPORT_H
+#define NUSPELL_EXPORT_H
+
+#ifdef NUSPELL_STATIC_DEFINE
+#  define NUSPELL_EXPORT
+#elif defined(_WIN32) || defined(__CYGWIN__)
+#  ifdef nuspell_EXPORTS // Define this only when building Nuspell as DLL on Windows, not when using the DLL.
+#    define NUSPELL_EXPORT __declspec(dllexport)
+#  else
+#    define NUSPELL_EXPORT __declspec(dllimport)
+#  endif
+#elif __GNUC__ >= 4
+#  define NUSPELL_EXPORT __attribute__((visibility("default")))
+#else
+#  define NUSPELL_EXPORT
+#endif
+
+#endif /* NUSPELL_EXPORT_H */

app/src/main/cpp/nuspell/suggester.hxx

@@ -0,0 +1,97 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef NUSPELL_SUGGESTER_HXX
+#define NUSPELL_SUGGESTER_HXX
+
+#include "checker.hxx"
+
+namespace nuspell {
+inline namespace v5 {
+
+struct NUSPELL_EXPORT Suggester : public Checker {
+
+	enum High_Quality_Sugs : bool {
+		ALL_LOW_QUALITY_SUGS = false,
+		HAS_HIGH_QUALITY_SUGS = true
+	};
+
+	auto suggest_priv(std::string_view input_word, List_Strings& out) const
+	    -> void;
+
+	auto suggest_low(std::string& word, List_Strings& out) const
+	    -> High_Quality_Sugs;
+
+	auto add_sug_if_correct(std::string& word, List_Strings& out) const
+	    -> bool;
+
+	auto uppercase_suggest(const std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto rep_suggest(std::string& word, List_Strings& out) const -> void;
+
+	auto try_rep_suggestion(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto max_attempts_for_long_alogs(std::string_view word) const -> size_t;
+
+	auto map_suggest(std::string& word, List_Strings& out) const -> void;
+
+	auto map_suggest(std::string& word, List_Strings& out, size_t i,
+	                 size_t& remaining_attempts) const -> void;
+
+	auto adjacent_swap_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto distant_swap_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto keyboard_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto extra_char_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto forgotten_char_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto move_char_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto bad_char_suggest(std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto doubled_two_chars_suggest(std::string& word,
+	                               List_Strings& out) const -> void;
+
+	auto two_words_suggest(const std::string& word, List_Strings& out) const
+	    -> void;
+
+	auto ngram_suggest(const std::string& word_u8, List_Strings& out) const
+	    -> void;
+
+	auto expand_root_word_for_ngram(Word_List::const_reference root,
+	                                std::string_view wrong,
+	                                List_Strings& expanded_list,
+	                                std::vector<bool>& cross_affix) const
+	    -> void;
+};
+
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_SUGGESTER_HXX

app/src/main/cpp/nuspell/unicode.hxx

@@ -0,0 +1,383 @@
+/* Copyright 2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef NUSPELL_UNICODE_HXX
+#define NUSPELL_UNICODE_HXX
+#include <string>
+#include <string_view>
+#include <unicode/utf16.h>
+#include <unicode/utf8.h>
+
+namespace nuspell {
+inline namespace v5 {
+
+// UTF-8, work on malformed
+
+inline constexpr auto u8_max_cp_length = U8_MAX_LENGTH;
+
+auto inline u8_is_cp_error(int32_t cp) -> bool { return cp < 0; }
+
+template <class Range>
+auto u8_advance_cp(const Range& str, size_t& i, int32_t& cp) -> void
+{
+	using std::size, std::data;
+#if U_ICU_VERSION_MAJOR_NUM <= 60
+	auto s_ptr = data(str);
+	int32_t idx = i;
+	int32_t len = size(str);
+	U8_NEXT(s_ptr, idx, len, cp);
+	i = idx;
+#else
+	auto len = size(str);
+	U8_NEXT(str, i, len, cp);
+#endif
+}
+
+template <class Range>
+auto u8_advance_index(const Range& str, size_t& i) -> void
+{
+	using std::size;
+	auto len = size(str);
+	U8_FWD_1(str, i, len);
+}
+
+template <class Range>
+auto u8_reverse_cp(const Range& str, size_t& i, int32_t& cp) -> void
+{
+	using std::size, std::data;
+	auto ptr = data(str);
+	int32_t idx = i;
+	U8_PREV(ptr, 0, idx, cp);
+	i = idx;
+}
+
+template <class Range>
+auto u8_reverse_index(const Range& str, size_t& i) -> void
+{
+	using std::size, std::data;
+	auto ptr = data(str);
+	int32_t idx = i;
+	U8_BACK_1(ptr, 0, idx);
+	i = idx;
+}
+
+template <class Range>
+auto u8_write_cp_and_advance(Range& buf, size_t& i, int32_t cp, bool& error)
+    -> void
+{
+	using std::size, std::data;
+#if U_ICU_VERSION_MAJOR_NUM <= 60
+	auto ptr = data(buf);
+	int32_t idx = i;
+	int32_t len = size(buf);
+	U8_APPEND(buf, idx, len, cp, error);
+	i = idx;
+#else
+	auto len = size(buf);
+	U8_APPEND(buf, i, len, cp, error);
+#endif
+}
+
+// UTF-8, valid
+
+template <class Range>
+auto valid_u8_advance_cp(const Range& str, size_t& i, char32_t& cp) -> void
+{
+	U8_NEXT_UNSAFE(str, i, cp);
+}
+
+template <class Range>
+auto valid_u8_advance_index(const Range& str, size_t& i) -> void
+{
+	U8_FWD_1_UNSAFE(str, i);
+}
+
+template <class Range>
+auto valid_u8_reverse_cp(const Range& str, size_t& i, char32_t& cp) -> void
+{
+	U8_PREV_UNSAFE(str, i, cp);
+}
+
+template <class Range>
+auto valid_u8_reverse_index(const Range& str, size_t& i) -> void
+{
+	U8_BACK_1_UNSAFE(str, i);
+}
+
+template <class Range>
+auto valid_u8_write_cp_and_advance(Range& buf, size_t& i, char32_t cp) -> void
+{
+	U8_APPEND_UNSAFE(buf, i, cp);
+}
+
+// UTF-16, work on malformed
+
+inline constexpr auto u16_max_cp_length = U16_MAX_LENGTH;
+
+auto inline u16_is_cp_error(int32_t cp) -> bool { return U_IS_SURROGATE(cp); }
+
+template <class Range>
+auto u16_advance_cp(const Range& str, size_t& i, int32_t& cp) -> void
+{
+	using std::size;
+	auto len = size(str);
+	U16_NEXT(str, i, len, cp);
+}
+
+template <class Range>
+auto u16_advance_index(const Range& str, size_t& i) -> void
+{
+	using std::size;
+	auto len = size(str);
+	U16_FWD_1(str, i, len);
+}
+
+template <class Range>
+auto u16_reverse_cp(const Range& str, size_t& i, int32_t& cp) -> void
+{
+	U16_PREV(str, 0, i, cp);
+}
+
+template <class Range>
+auto u16_reverse_index(const Range& str, size_t& i) -> void
+{
+	U16_BACK_1(str, 0, i);
+}
+
+template <class Range>
+auto u16_write_cp_and_advance(Range& buf, size_t& i, int32_t cp, bool& error)
+    -> void
+{
+	using std::size;
+	auto len = size(buf);
+	U16_APPEND(buf, i, len, cp, error);
+}
+
+// UTF-16, valid
+
+template <class Range>
+auto valid_u16_advance_cp(const Range& str, size_t& i, char32_t& cp) -> void
+{
+	U16_NEXT_UNSAFE(str, i, cp);
+}
+
+template <class Range>
+auto valid_u16_advance_index(const Range& str, size_t& i) -> void
+{
+	U16_FWD_1_UNSAFE(str, i);
+}
+
+template <class Range>
+auto valid_u16_reverse_cp(const Range& str, size_t& i, char32_t& cp) -> void
+{
+	U16_PREV_UNSAFE(str, i, cp);
+}
+
+template <class Range>
+auto valid_u16_reverse_index(const Range& str, size_t& i) -> void
+{
+	U16_BACK_1_UNSAFE(str, i);
+}
+
+template <class Range>
+auto valid_u16_write_cp_and_advance(Range& buf, size_t& i, char32_t cp) -> void
+{
+	U16_APPEND_UNSAFE(buf, i, cp);
+}
+
+// higer level funcs
+
+struct U8_CP_Pos {
+	size_t begin_i = 0;
+	size_t end_i = begin_i;
+};
+
+class U8_Encoded_CP {
+	char d[u8_max_cp_length];
+	int sz;
+
+      public:
+	explicit U8_Encoded_CP(std::string_view str, U8_CP_Pos pos)
+	    : sz(pos.end_i - pos.begin_i)
+	{
+		auto i = sz;
+		auto j = pos.end_i;
+		auto max_len = 4;
+		do {
+			d[--i] = str[--j];
+		} while (i && --max_len);
+	}
+	U8_Encoded_CP(char32_t cp)
+	{
+		size_t z = 0;
+		valid_u8_write_cp_and_advance(d, z, cp);
+		sz = z;
+	}
+	auto size() const noexcept -> size_t { return sz; }
+	auto data() const noexcept -> const char* { return d; }
+	operator std::string_view() const noexcept
+	{
+		return std::string_view(data(), size());
+	}
+	auto copy_to(std::string& str, size_t j) const
+	{
+		auto i = sz;
+		j += sz;
+		auto max_len = 4;
+		do {
+			str[--j] = d[--i];
+		} while (i && --max_len);
+	}
+};
+
+auto inline u8_swap_adjacent_cp(std::string& str, size_t i1, size_t i2,
+                                size_t i3) -> size_t
+{
+	auto cp1 = U8_Encoded_CP(str, {i1, i2});
+	auto cp2 = U8_Encoded_CP(str, {i2, i3});
+	auto new_i2 = i1 + std::size(cp2);
+	cp1.copy_to(str, new_i2);
+	cp2.copy_to(str, i1);
+	return new_i2;
+}
+
+auto inline u8_swap_cp(std::string& str, U8_CP_Pos pos1, U8_CP_Pos pos2)
+    -> std::pair<size_t, size_t>
+{
+	using std::size;
+	auto cp1 = U8_Encoded_CP(str, pos1);
+	auto cp2 = U8_Encoded_CP(str, pos2);
+	auto new_p1_end_i = pos1.begin_i + size(cp2);
+	auto new_p2_begin_i = pos2.end_i - size(cp1);
+	std::char_traits<char>::move(&str[new_p1_end_i], &str[pos1.end_i],
+	                             pos2.begin_i - pos1.end_i);
+	cp2.copy_to(str, pos1.begin_i);
+	cp1.copy_to(str, new_p2_begin_i);
+	return {new_p1_end_i, new_p2_begin_i};
+}
+
+// bellow go func without out-parametars
+
+// UTF-8, can be malformed, no out-parametars
+
+struct Idx_And_Next_CP {
+	size_t end_i;
+	int32_t cp;
+};
+
+struct Idx_And_Prev_CP {
+	size_t begin_i;
+	int32_t cp;
+};
+
+struct Write_CP_Idx_and_Error {
+	size_t end_i;
+	bool error;
+};
+
+template <class Range>
+[[nodiscard]] auto u8_next_cp(const Range& str, size_t i) -> Idx_And_Next_CP
+{
+	int32_t cp;
+	u8_advance_cp(str, i, cp);
+	return {i, cp};
+}
+
+template <class Range>
+[[nodiscard]] auto u8_next_index(const Range& str, size_t i) -> size_t
+{
+	u8_advance_index(str, i);
+	return i;
+}
+
+template <class Range>
+[[nodiscard]] auto u8_prev_cp(const Range& str, size_t i) -> Idx_And_Prev_CP
+{
+	int32_t cp;
+	u8_reverse_cp(str, i, cp);
+	return {i, cp};
+}
+
+template <class Range>
+[[nodiscard]] auto u8_prev_index(const Range& str, size_t i) -> size_t
+{
+	u8_reverse_index(str, i);
+	return i;
+}
+
+template <class Range>
+[[nodiscard]] auto u8_write_cp(Range& buf, size_t i, int32_t cp)
+    -> Write_CP_Idx_and_Error
+{
+	bool err;
+	u8_write_cp_and_advance(buf, i, cp, err);
+	return {i, err};
+}
+
+// UTF-8, valid, no out-parametars
+
+struct Idx_And_Next_CP_Valid {
+	size_t end_i;
+	char32_t cp;
+};
+
+struct Idx_And_Prev_CP_Valid {
+	size_t begin_i;
+	char32_t cp;
+};
+
+template <class Range>
+[[nodiscard]] auto valid_u8_next_cp(const Range& str, size_t i)
+    -> Idx_And_Next_CP_Valid
+{
+	char32_t cp;
+	valid_u8_advance_cp(str, i, cp);
+	return {i, cp};
+}
+
+template <class Range>
+[[nodiscard]] auto valid_u8_next_index(const Range& str, size_t i) -> size_t
+{
+	valid_u8_advance_index(str, i);
+	return i;
+}
+
+template <class Range>
+[[nodiscard]] auto valid_u8_prev_cp(const Range& str, size_t i)
+    -> Idx_And_Prev_CP_Valid
+{
+	char32_t cp;
+	valid_u8_reverse_cp(str, i, cp);
+	return {i, cp};
+}
+
+template <class Range>
+[[nodiscard]] auto valid_u8_prev_index(const Range& str, size_t i) -> size_t
+{
+	valid_u8_reverse_index(str, i);
+	return i;
+}
+
+template <class Range>
+[[nodiscard]] auto valid_u8_write_cp(Range& buf, size_t i, int32_t cp) -> size_t
+{
+	valid_u8_write_cp_and_advance(buf, i, cp);
+	return i;
+}
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_UNICODE_HXX

app/src/main/cpp/nuspell/utils.cxx

@@ -0,0 +1,465 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "utils.hxx"
+#include "unicode.hxx"
+
+#include <algorithm>
+#include <limits>
+
+#include <unicode/uchar.h>
+#include <unicode/ucnv.h>
+#include <unicode/unistr.h>
+#include <unicode/ustring.h>
+
+#if ' ' != 32 || '.' != 46 || 'A' != 65 || 'Z' != 90 || 'a' != 97 || 'z' != 122
+#error "Basic execution character set is not ASCII"
+#endif
+
+using namespace std;
+
+namespace nuspell {
+inline namespace v5 {
+
+template <class SepT>
+static auto& split_on_any_of_low(std::string_view s, const SepT& sep,
+                                 std::vector<std::string>& out)
+{
+	size_t i1 = 0;
+	size_t i2;
+	do {
+		i2 = s.find_first_of(sep, i1);
+		out.emplace_back(s.substr(i1, i2 - i1));
+		i1 = i2 + 1; // we can only add +1 if separator is single char.
+
+		// i2 gets s.npos after the last separator.
+		// Length of i2-i1 will always go past the end. That is defined.
+	} while (i2 != s.npos);
+	return out;
+}
+
+/**
+ * @internal
+ * @brief Splits string on single char seperator.
+ *
+ * Consecutive separators are treated as separate and will emit empty strings.
+ *
+ * @param s string to split.
+ * @param sep char that acts as separator to split on.
+ * @param out vector where separated strings are appended.
+ * @return @p out.
+ */
+auto split(std::string_view s, char sep, std::vector<std::string>& out)
+    -> std::vector<std::string>&
+{
+	return split_on_any_of_low(s, sep, out);
+}
+
+/**
+ * @internal
+ * @brief Splits string on set of single char seperators.
+ *
+ * Consecutive separators are treated as separate and will emit empty strings.
+ *
+ * @param s string to split.
+ * @param sep seperator(s) to split on.
+ * @param out vector where separated strings are appended.
+ * @return @p out.
+ */
+auto split_on_any_of(std::string_view s, const char* sep,
+                     std::vector<std::string>& out) -> std::vector<std::string>&
+{
+	return split_on_any_of_low(s, sep, out);
+}
+
+auto utf32_to_utf8(std::u32string_view in, std::string& out) -> void
+{
+	out.clear();
+	for (size_t i = 0; i != size(in); ++i) {
+		auto cp = in[i];
+		auto enc_cp = U8_Encoded_CP(cp);
+		out += enc_cp;
+	}
+}
+auto utf32_to_utf8(std::u32string_view in) -> std::string
+{
+	auto out = string();
+	utf32_to_utf8(in, out);
+	return out;
+}
+
+auto valid_utf8_to_32(std::string_view in, std::u32string& out) -> void
+{
+	out.clear();
+	for (size_t i = 0; i != size(in);) {
+		char32_t cp;
+		valid_u8_advance_cp(in, i, cp);
+		out.push_back(cp);
+	}
+}
+auto valid_utf8_to_32(std::string_view in) -> std::u32string
+{
+	auto out = u32string();
+	valid_utf8_to_32(in, out);
+	return out;
+}
+
+auto utf8_to_16(std::string_view in) -> std::u16string
+{
+	auto out = u16string();
+	utf8_to_16(in, out);
+	return out;
+}
+
+bool utf8_to_16(std::string_view in, std::u16string& out)
+{
+	int32_t len;
+	auto err = U_ZERO_ERROR;
+	u_strFromUTF8(data(out), size(out), &len, data(in), size(in), &err);
+	out.resize(len);
+	if (err == U_BUFFER_OVERFLOW_ERROR) {
+		err = U_ZERO_ERROR;
+		u_strFromUTF8(data(out), size(out), &len, data(in), size(in),
+		              &err);
+	}
+	if (U_SUCCESS(err))
+		return true;
+	out.clear();
+	return false;
+}
+
+bool validate_utf8(string_view s)
+{
+	auto err = U_ZERO_ERROR;
+	u_strFromUTF8(nullptr, 0, nullptr, data(s), size(s), &err);
+	if (err == U_INVALID_CHAR_FOUND)
+		return false;
+	return err == U_BUFFER_OVERFLOW_ERROR || U_SUCCESS(err);
+}
+
+auto static is_ascii(char c) -> bool
+{
+	return static_cast<unsigned char>(c) <= 127;
+}
+
+auto is_all_ascii(std::string_view s) -> bool
+{
+	return all_of(begin(s), end(s), is_ascii);
+}
+
+auto static widen_latin1(char c) -> char16_t
+{
+	return static_cast<unsigned char>(c);
+}
+
+auto latin1_to_ucs2(std::string_view s) -> std::u16string
+{
+	u16string ret;
+	latin1_to_ucs2(s, ret);
+	return ret;
+}
+auto latin1_to_ucs2(std::string_view s, std::u16string& out) -> void
+{
+	out.resize(s.size());
+	transform(begin(s), end(s), begin(out), widen_latin1);
+}
+
+auto static is_surrogate_pair(char16_t c) -> bool
+{
+	return 0xD800 <= c && c <= 0xDFFF;
+}
+auto is_all_bmp(std::u16string_view s) -> bool
+{
+	return none_of(begin(s), end(s), is_surrogate_pair);
+}
+
+auto to_upper_ascii(std::string& s) -> void
+{
+	auto& char_type = use_facet<ctype<char>>(locale::classic());
+	char_type.toupper(begin_ptr(s), end_ptr(s));
+}
+
+auto static utf32_to_icu(u32string_view in) -> icu::UnicodeString
+{
+	static_assert(sizeof(UChar32) == sizeof(char32_t));
+	return icu::UnicodeString::fromUTF32(
+	    reinterpret_cast<const UChar32*>(in.data()), in.size());
+}
+auto static icu_to_utf32(const icu::UnicodeString& in, std::u32string& out)
+    -> bool
+{
+	out.resize(in.length());
+	auto err = U_ZERO_ERROR;
+	auto len =
+	    in.toUTF32(reinterpret_cast<UChar32*>(out.data()), out.size(), err);
+	if (U_SUCCESS(err)) {
+		out.erase(len);
+		return true;
+	}
+	out.clear();
+	return false;
+}
+
+auto to_upper(std::string_view in, const icu::Locale& loc) -> std::string
+{
+	auto out = std::string();
+	to_upper(in, loc, out);
+	return out;
+}
+auto to_title(std::string_view in, const icu::Locale& loc) -> std::string
+{
+	auto out = std::string();
+	to_title(in, loc, out);
+	return out;
+}
+auto to_lower(std::string_view in, const icu::Locale& loc) -> std::string
+{
+	auto out = std::string();
+	to_lower(in, loc, out);
+	return out;
+}
+
+auto to_upper(string_view in, const icu::Locale& loc, string& out) -> void
+{
+	auto sp = icu::StringPiece(data(in), size(in));
+	auto us = icu::UnicodeString::fromUTF8(sp);
+	us.toUpper(loc);
+	out.clear();
+	us.toUTF8String(out);
+}
+auto to_title(string_view in, const icu::Locale& loc, string& out) -> void
+{
+	auto sp = icu::StringPiece(data(in), size(in));
+	auto us = icu::UnicodeString::fromUTF8(sp);
+	us.toTitle(nullptr, loc);
+	out.clear();
+	us.toUTF8String(out);
+}
+auto to_lower(u32string_view in, const icu::Locale& loc, u32string& out) -> void
+{
+	auto us = utf32_to_icu(in);
+	us.toLower(loc);
+	icu_to_utf32(us, out);
+}
+auto to_lower(string_view in, const icu::Locale& loc, string& out) -> void
+{
+	auto sp = icu::StringPiece(data(in), size(in));
+	auto us = icu::UnicodeString::fromUTF8(sp);
+	us.toLower(loc);
+	out.clear();
+	us.toUTF8String(out);
+}
+
+auto to_lower_char_at(std::string& s, size_t i, const icu::Locale& loc) -> void
+{
+	auto cp = valid_u8_next_cp(s, i);
+	auto us = icu::UnicodeString(UChar32(cp.cp));
+	us.toLower(loc);
+	auto u8_low = string();
+	us.toUTF8String(u8_low);
+	s.replace(i, cp.end_i - i, u8_low);
+}
+auto to_title_char_at(std::string& s, size_t i, const icu::Locale& loc) -> void
+{
+	auto cp = valid_u8_next_cp(s, i);
+	auto us = icu::UnicodeString(UChar32(cp.cp));
+	us.toTitle(nullptr, loc);
+	auto u8_title = string();
+	us.toUTF8String(u8_title);
+	s.replace(i, cp.end_i - i, u8_title);
+}
+
+/**
+ * @internal
+ * @brief Determines casing (capitalization) type for a word.
+ *
+ * Casing is sometimes referred to as capitalization.
+ *
+ * @param s word.
+ * @return The casing type.
+ */
+auto classify_casing(string_view s) -> Casing
+{
+	size_t upper = 0;
+	size_t lower = 0;
+	for (size_t i = 0; i != size(s);) {
+		char32_t c;
+		valid_u8_advance_cp(s, i, c);
+		if (u_isupper(c))
+			upper++;
+		else if (u_islower(c))
+			lower++;
+		// else neutral
+	}
+	if (upper == 0)               // all lowercase, maybe with some neutral
+		return Casing::SMALL; // most common case
+
+	auto first_cp = valid_u8_next_cp(s, 0);
+	auto first_capital = u_isupper(first_cp.cp);
+	if (first_capital && upper == 1)
+		return Casing::INIT_CAPITAL; // second most common
+
+	if (lower == 0)
+		return Casing::ALL_CAPITAL;
+
+	if (first_capital)
+		return Casing::PASCAL;
+	else
+		return Casing::CAMEL;
+}
+
+/**
+ * @internal
+ * @brief Check if word[i] or word[i-1] are uppercase
+ *
+ * Check if the two chars are alphabetic and at least one of them is in
+ * uppercase.
+ *
+ * @return true if at least one is uppercase, false otherwise.
+ */
+auto has_uppercase_at_compound_word_boundary(string_view word, size_t i) -> bool
+{
+	auto cp = valid_u8_next_cp(word, i);
+	auto cp_prev = valid_u8_prev_cp(word, i);
+	if (u_isupper(cp.cp)) {
+		if (u_isalpha(cp_prev.cp))
+			return true;
+	}
+	else if (u_isupper(cp_prev.cp) && u_isalpha(cp.cp))
+		return true;
+	return false;
+}
+
+Encoding_Converter::Encoding_Converter(const char* enc)
+{
+	auto err = UErrorCode();
+	cnv = ucnv_open(enc, &err);
+}
+
+Encoding_Converter::~Encoding_Converter()
+{
+	if (cnv)
+		ucnv_close(cnv);
+}
+
+Encoding_Converter::Encoding_Converter(const Encoding_Converter& other)
+{
+	auto err = UErrorCode();
+	cnv = ucnv_safeClone(other.cnv, nullptr, nullptr, &err);
+}
+
+auto Encoding_Converter::operator=(const Encoding_Converter& other)
+    -> Encoding_Converter&
+{
+	this->~Encoding_Converter();
+	auto err = UErrorCode();
+	cnv = ucnv_safeClone(other.cnv, nullptr, nullptr, &err);
+	return *this;
+}
+
+auto Encoding_Converter::to_utf8(string_view in, string& out) -> bool
+{
+	if (ucnv_getType(cnv) == UCNV_UTF8) {
+		if (validate_utf8(in)) {
+			out = in;
+			return true;
+		}
+		else {
+			out.clear();
+			return false;
+		}
+	}
+	auto err = U_ZERO_ERROR;
+	auto len = ucnv_toAlgorithmic(UCNV_UTF8, cnv, out.data(), out.size(),
+	                              in.data(), in.size(), &err);
+	out.resize(len);
+	if (err == U_BUFFER_OVERFLOW_ERROR) {
+		err = U_ZERO_ERROR;
+		ucnv_toAlgorithmic(UCNV_UTF8, cnv, out.data(), out.size(),
+		                   in.data(), in.size(), &err);
+	}
+	return U_SUCCESS(err);
+}
+
+auto replace_ascii_char(string& s, char from, char to) -> void
+{
+	for (auto i = s.find(from); i != s.npos; i = s.find(from, i + 1)) {
+		s[i] = to;
+	}
+}
+
+auto erase_chars(string& s, string_view erase_chars) -> void
+{
+	if (erase_chars.empty())
+		return;
+	for (size_t i = 0, next_i = 0; i != size(s); i = next_i) {
+		valid_u8_advance_index(s, next_i);
+		auto enc_cp = string_view(&s[i], next_i - i);
+		if (erase_chars.find(enc_cp) != erase_chars.npos) {
+			s.erase(i, next_i - i);
+			next_i = i;
+		}
+	}
+	return;
+}
+
+/**
+ * @internal
+ * @brief Tests if word is a number.
+ *
+ * Allow numbers with dot ".", dash "-" or comma "," inbetween the digits, but
+ * forbids double separators such as "..", "--" and ".,".
+ */
+auto is_number(string_view s) -> bool
+{
+	if (s.empty())
+		return false;
+
+	auto it = begin(s);
+	if (s[0] == '-')
+		++it;
+	while (it != end(s)) {
+		auto next = std::find_if(
+		    it, end(s), [](auto c) { return c < '0' || c > '9'; });
+		if (next == it)
+			return false;
+		if (next == end(s))
+			return true;
+		it = next;
+		auto c = *it;
+		if (c == '.' || c == ',' || c == '-')
+			++it;
+		else
+			return false;
+	}
+	return false;
+}
+
+auto count_appereances_of(string_view haystack, string_view needles) -> size_t
+{
+	auto ret = size_t(0);
+	for (size_t i = 0, next_i = 0; i != size(haystack); i = next_i) {
+		valid_u8_advance_index(haystack, next_i);
+		auto enc_cp = string_view(&haystack[i], next_i - i);
+		ret += needles.find(enc_cp) != needles.npos;
+	}
+	return ret;
+}
+
+} // namespace v5
+} // namespace nuspell

app/src/main/cpp/nuspell/utils.hxx

@@ -0,0 +1,228 @@
+/* Copyright 2016-2021 Dimitrij Mijoski
+ *
+ * This file is part of Nuspell.
+ *
+ * Nuspell is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * Nuspell is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with Nuspell.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef NUSPELL_UTILS_HXX
+#define NUSPELL_UTILS_HXX
+
+#include "nuspell_export.h"
+
+#include <clocale>
+#include <locale>
+#include <string>
+#include <string_view>
+#include <vector>
+
+#if !defined(_WIN32) &&                                                        \
+    (defined(__unix__) || defined(__unix) ||                                   \
+     (defined(__APPLE__) && defined(__MACH__)) || defined(__HAIKU__))
+#include <unistd.h>
+#endif
+
+#include <unicode/locid.h>
+
+#ifdef __GNUC__
+#define likely(expr) __builtin_expect(!!(expr), 1)
+#define unlikely(expr) __builtin_expect(!!(expr), 0)
+#else
+#define likely(expr) (expr)
+#define unlikely(expr) (expr)
+#endif
+
+struct UConverter; // unicode/ucnv.h
+
+namespace nuspell {
+inline namespace v5 {
+
+auto split(std::string_view s, char sep, std::vector<std::string>& out)
+    -> std::vector<std::string>&;
+NUSPELL_EXPORT auto split_on_any_of(std::string_view s, const char* sep,
+                                    std::vector<std::string>& out)
+    -> std::vector<std::string>&;
+
+NUSPELL_EXPORT auto utf32_to_utf8(std::u32string_view in, std::string& out)
+    -> void;
+NUSPELL_EXPORT auto utf32_to_utf8(std::u32string_view in) -> std::string;
+
+auto valid_utf8_to_32(std::string_view in, std::u32string& out) -> void;
+auto valid_utf8_to_32(std::string_view in) -> std::u32string;
+
+auto utf8_to_16(std::string_view in) -> std::u16string;
+auto utf8_to_16(std::string_view in, std::u16string& out) -> bool;
+
+auto validate_utf8(std::string_view s) -> bool;
+
+NUSPELL_EXPORT auto is_all_ascii(std::string_view s) -> bool;
+
+NUSPELL_EXPORT auto latin1_to_ucs2(std::string_view s) -> std::u16string;
+auto latin1_to_ucs2(std::string_view s, std::u16string& out) -> void;
+
+NUSPELL_EXPORT auto is_all_bmp(std::u16string_view s) -> bool;
+
+auto to_upper_ascii(std::string& s) -> void;
+
+[[nodiscard]] NUSPELL_EXPORT auto to_upper(std::string_view in,
+                                           const icu::Locale& loc)
+    -> std::string;
+[[nodiscard]] NUSPELL_EXPORT auto to_title(std::string_view in,
+                                           const icu::Locale& loc)
+    -> std::string;
+[[nodiscard]] NUSPELL_EXPORT auto to_lower(std::string_view in,
+                                           const icu::Locale& loc)
+    -> std::string;
+
+auto to_upper(std::string_view in, const icu::Locale& loc, std::string& out)
+    -> void;
+auto to_title(std::string_view in, const icu::Locale& loc, std::string& out)
+    -> void;
+auto to_lower(std::u32string_view in, const icu::Locale& loc,
+              std::u32string& out) -> void;
+auto to_lower(std::string_view in, const icu::Locale& loc, std::string& out)
+    -> void;
+auto to_lower_char_at(std::string& s, size_t i, const icu::Locale& loc) -> void;
+auto to_title_char_at(std::string& s, size_t i, const icu::Locale& loc) -> void;
+
+/**
+ * @internal
+ * @brief Enum that identifies the casing type of a word.
+ *
+ * Neutral characters like numbers are ignored, so "abc" and "abc123abc" are
+ * both classified as small.
+ */
+enum class Casing : char {
+	SMALL,
+	INIT_CAPITAL,
+	ALL_CAPITAL,
+	CAMEL /**< @internal camelCase i.e. mixed case with first small */,
+	PASCAL /**< @internal  PascalCase i.e. mixed case with first capital */
+};
+
+NUSPELL_EXPORT auto classify_casing(std::string_view s) -> Casing;
+
+auto has_uppercase_at_compound_word_boundary(std::string_view word, size_t i)
+    -> bool;
+
+class Encoding_Converter {
+	UConverter* cnv = nullptr;
+
+      public:
+	Encoding_Converter() = default;
+	explicit Encoding_Converter(const char* enc);
+	explicit Encoding_Converter(const std::string& enc)
+	    : Encoding_Converter(enc.c_str())
+	{
+	}
+	~Encoding_Converter();
+	Encoding_Converter(const Encoding_Converter& other);
+	Encoding_Converter(Encoding_Converter&& other) noexcept
+	{
+		cnv = other.cnv;
+		cnv = nullptr;
+	}
+	auto operator=(const Encoding_Converter& other) -> Encoding_Converter&;
+	auto operator=(Encoding_Converter&& other) noexcept
+	    -> Encoding_Converter&
+	{
+		std::swap(cnv, other.cnv);
+		return *this;
+	}
+	auto to_utf8(std::string_view in, std::string& out) -> bool;
+	auto valid() -> bool { return cnv != nullptr; }
+};
+
+//#if _POSIX_VERSION >= 200809L
+#if defined(_POSIX_VERSION) && !defined(__NetBSD__) && !defined(__HAIKU__)
+class Setlocale_To_C_In_Scope {
+	locale_t old_loc = nullptr;
+
+      public:
+	Setlocale_To_C_In_Scope()
+	    : old_loc{uselocale(newlocale(0, "C", nullptr))}
+	{
+	}
+	~Setlocale_To_C_In_Scope()
+	{
+		auto new_loc = uselocale(old_loc);
+		if (new_loc != old_loc)
+			freelocale(new_loc);
+	}
+	Setlocale_To_C_In_Scope(const Setlocale_To_C_In_Scope&) = delete;
+};
+#else
+class Setlocale_To_C_In_Scope {
+	std::string old_name;
+#ifdef _WIN32
+	int old_per_thread;
+#endif
+      public:
+	Setlocale_To_C_In_Scope() : old_name(setlocale(LC_ALL, nullptr))
+	{
+#ifdef _WIN32
+		old_per_thread = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
+#endif
+		auto x = setlocale(LC_ALL, "C");
+		if (!x)
+			old_name.clear();
+	}
+	~Setlocale_To_C_In_Scope()
+	{
+#ifdef _WIN32
+		_configthreadlocale(old_per_thread);
+		if (old_per_thread == _ENABLE_PER_THREAD_LOCALE)
+#endif
+		{
+			if (!old_name.empty())
+				setlocale(LC_ALL, old_name.c_str());
+		}
+	}
+	Setlocale_To_C_In_Scope(const Setlocale_To_C_In_Scope&) = delete;
+};
+#endif
+
+auto replace_ascii_char(std::string& s, char from, char to) -> void;
+auto erase_chars(std::string& s, std::string_view erase_chars) -> void;
+NUSPELL_EXPORT auto is_number(std::string_view s) -> bool;
+auto count_appereances_of(std::string_view haystack, std::string_view needles)
+    -> size_t;
+
+auto inline begins_with(std::string_view haystack, std::string_view needle)
+    -> bool
+{
+	return haystack.compare(0, needle.size(), needle) == 0;
+}
+
+auto inline ends_with(std::string_view haystack, std::string_view needle)
+    -> bool
+{
+	return haystack.size() >= needle.size() &&
+	       haystack.compare(haystack.size() - needle.size(), needle.size(),
+	                        needle) == 0;
+}
+
+template <class T>
+auto begin_ptr(T& x)
+{
+	return x.data();
+}
+template <class T>
+auto end_ptr(T& x)
+{
+	return x.data() + x.size();
+}
+} // namespace v5
+} // namespace nuspell
+#endif // NUSPELL_UTILS_HXX

app/src/main/cpp/utils/CMakeLists.txt

@@ -0,0 +1,12 @@
+add_library(
+        # Name
+        utils
+
+        # Headers
+        jni_utils.h
+        log.h
+
+        # Sources
+        jni_utils.cpp
+        log.cpp
+)

app/src/main/cpp/utils/jni_utils.cpp

@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "jni_utils.h"
+#include "log.h"
+
+std::string utils::j2std_string(JNIEnv *env, jobject jStr) {
+    auto cStr = reinterpret_cast<const char *>(env->GetDirectBufferAddress(jStr));
+    auto size = env->GetDirectBufferCapacity(jStr);
+    std::string stdStr(cStr, size);
+    log_debug("spell j2s", stdStr);
+    return stdStr;
+}
+
+jobject utils::std2j_string(JNIEnv *env, const std::string& stdStr) {
+    log_debug("spell s2j", stdStr);
+    size_t byteCount = stdStr.length();
+    auto cStr = stdStr.c_str();
+    auto buffer = env->NewDirectByteBuffer((void *) cStr, byteCount);
+    return buffer;
+}

app/src/main/cpp/utils/jni_utils.h

@@ -0,0 +1,30 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef FLORISBOARD_JNI_UTILS_H
+#define FLORISBOARD_JNI_UTILS_H
+
+#include <jni.h>
+#include <string>
+
+namespace utils {
+
+std::string j2std_string(JNIEnv *env, jobject jStr);
+jobject std2j_string(JNIEnv *env, const std::string& in);
+
+} // namespace utils
+
+#endif // FLORISBOARD_JNI_UTILS_H

app/src/main/cpp/utils/log.cpp

@@ -0,0 +1,81 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <android/log.h>
+#include <unistd.h>
+#include "log.h"
+
+void utils::log_debug(const std::string &tag, const std::string &msg) {
+    __android_log_print(ANDROID_LOG_DEBUG, tag.c_str(), "%s", msg.c_str());
+}
+
+void utils::log_info(const std::string &tag, const std::string &msg) {
+    __android_log_print(ANDROID_LOG_INFO, tag.c_str(), "%s", msg.c_str());
+}
+
+void utils::log_warning(const std::string &tag, const std::string &msg) {
+    __android_log_print(ANDROID_LOG_WARN, tag.c_str(), "%s", msg.c_str());
+}
+
+void utils::log_error(const std::string &tag, const std::string &msg) {
+    __android_log_print(ANDROID_LOG_ERROR, tag.c_str(), "%s", msg.c_str());
+}
+
+void utils::log_wtf(const std::string &tag, const std::string &msg) {
+    __android_log_print(ANDROID_LOG_FATAL, tag.c_str(), "%s", msg.c_str());
+}
+
+/**
+ * Code below taken from here:
+ *  https://codelab.wordpress.com/2014/11/03/how-to-use-standard-output-streams-for-logging-in-android-apps/
+ */
+static int pfd[2];
+static pthread_t thr;
+static const char *tag = "myapp";
+static bool already_started = false;
+
+static void *thread_func(void*) {
+    ssize_t rdsz;
+    char buf[2048];
+    while ((rdsz = read(pfd[0], buf, sizeof buf - 1)) > 0) {
+        if (buf[rdsz - 1] == '\n') --rdsz;
+        buf[rdsz] = 0;  /* add null-terminator */
+        __android_log_write(ANDROID_LOG_DEBUG, tag, buf);
+    }
+    return nullptr;
+}
+
+int utils::start_stdout_stderr_logger(const char *app_name) {
+    if (already_started) return 0;
+    already_started = true;
+    tag = app_name;
+
+    /* make stdout line-buffered and stderr unbuffered */
+    setvbuf(stdout, nullptr, _IOLBF, 0);
+    setvbuf(stderr, nullptr, _IONBF, 0);
+
+    /* create the pipe and redirect stdout and stderr */
+    pipe(pfd);
+    dup2(pfd[1], 1);
+    dup2(pfd[1], 2);
+
+    /* spawn the logging thread */
+    if (pthread_create(&thr, nullptr, thread_func, nullptr) != 0) {
+        return -1;
+    }
+    pthread_detach(thr);
+    return 0;
+}

app/src/main/cpp/utils/log.h

@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef FLORISBOARD_LOG_H
+#define FLORISBOARD_LOG_H
+
+#include <string>
+
+namespace utils {
+
+void log_debug(const std::string& tag, const std::string& msg);
+void log_info(const std::string& tag, const std::string& msg);
+void log_warning(const std::string& tag, const std::string& msg);
+void log_error(const std::string& tag, const std::string& msg);
+void log_wtf(const std::string& tag, const std::string& msg);
+
+int start_stdout_stderr_logger(const char *app_name);
+
+} // namespace utils
+
+#endif // FLORISBOARD_LOG_H

app/src/main/java/dev/patrickgold/florisboard/FlorisApplication.kt

@@ -17,32 +17,105 @@
 package dev.patrickgold.florisboard
 
 import android.app.Application
+import android.content.BroadcastReceiver
+import android.content.Context
+import android.content.Intent
+import android.content.IntentFilter
+import android.os.Build
+import androidx.core.os.UserManagerCompat
+import dev.patrickgold.florisboard.common.NativeStr
+import dev.patrickgold.florisboard.common.toNativeStr
 import dev.patrickgold.florisboard.crashutility.CrashUtility
 import dev.patrickgold.florisboard.debug.Flog
 import dev.patrickgold.florisboard.debug.LogTopic
+import dev.patrickgold.florisboard.debug.flogError
+import dev.patrickgold.florisboard.debug.flogInfo
 import dev.patrickgold.florisboard.ime.core.Preferences
 import dev.patrickgold.florisboard.ime.core.SubtypeManager
 import dev.patrickgold.florisboard.ime.dictionary.DictionaryManager
-import dev.patrickgold.florisboard.ime.extension.AssetManager
+import dev.patrickgold.florisboard.ime.spelling.SpellingManager
 import dev.patrickgold.florisboard.ime.theme.ThemeManager
-import kotlinx.coroutines.CoroutineScope
-import kotlinx.coroutines.MainScope
+import dev.patrickgold.florisboard.res.AssetManager
+import dev.patrickgold.florisboard.res.FlorisRef
 import timber.log.Timber
+import java.io.File
+import kotlin.Exception
 
 @Suppress("unused")
-class FlorisApplication : Application(), CoroutineScope by MainScope() {
+class FlorisApplication : Application() {
+    companion object {
+        private const val ICU_DATA_ASSET_PATH = "icu/icudt69l.dat"
+
+        private external fun nativeInitICUData(path: NativeStr): Int
+
+        init {
+            try {
+                System.loadLibrary("florisboard-native")
+            } catch (_: Exception) {
+            }
+        }
+    }
+
     override fun onCreate() {
         super.onCreate()
-        if (BuildConfig.DEBUG) {
-            Timber.plant(Timber.DebugTree())
+        try {
+            if (BuildConfig.DEBUG) {
+                Timber.plant(Timber.DebugTree())
+            }
+            Flog.install(
+                applicationContext = this,
+                isFloggingEnabled = BuildConfig.DEBUG,
+                flogTopics = LogTopic.ALL,
+                flogLevels = Flog.LEVEL_ALL,
+                flogOutputs = Flog.OUTPUT_CONSOLE
+            )
+            initICU()
+            CrashUtility.install(this)
+            val prefs = Preferences.initDefault(this)
+            val assetManager = AssetManager.init(this)
+            SpellingManager.init(this, FlorisRef.assets("ime/spelling/config.json"))
+            SubtypeManager.init(this)
+            DictionaryManager.init(this)
+            ThemeManager.init(this, assetManager)
+            prefs.initDefaultPreferences()
+        } catch (e: Exception) {
+            CrashUtility.stageException(e)
+            return
         }
-        Flog.install(
-            applicationContext = this,
-            isFloggingEnabled = BuildConfig.DEBUG,
-            flogTopics = LogTopic.ALL,
-            flogLevels = Flog.LEVEL_ALL,
-            flogOutputs = Flog.OUTPUT_CONSOLE
-        )
+
+        /*Register a receiver so user config can be applied once device protracted storage is available*/
+        if(!UserManagerCompat.isUserUnlocked(this) && Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
+            registerReceiver(BootComplete(), IntentFilter(Intent.ACTION_USER_UNLOCKED))
+        }
+    }
+
+    fun initICU(): Boolean {
+        try {
+            val context = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
+                createDeviceProtectedStorageContext()
+            } else {
+                this
+            }
+            val androidAssetManager = context.assets ?: return false
+            val dstDataFile = File(context.cacheDir, "icudt.dat")
+            dstDataFile.outputStream().use { os ->
+                androidAssetManager.open(ICU_DATA_ASSET_PATH).use { it.copyTo(os) }
+            }
+            val status = nativeInitICUData(dstDataFile.absolutePath.toNativeStr())
+            return if (status != 0) {
+                flogError { "Native ICU data initializing failed with error code $status!" }
+                false
+            } else {
+                flogInfo { "Successfully loaded ICU data!" }
+                true
+            }
+        } catch (e: Exception) {
+            flogError { e.toString() }
+            return false
+        }
+    }
+
+    fun init() {
         CrashUtility.install(this)
         val prefs = Preferences.initDefault(this)
         val assetManager = AssetManager.init(this)
@@ -51,4 +124,18 @@ class FlorisApplication : Application(), CoroutineScope by MainScope() {
         ThemeManager.init(this, assetManager)
         prefs.initDefaultPreferences()
     }
+
+    private class BootComplete : BroadcastReceiver() {
+
+        override fun onReceive(context: Context?, intent: Intent?) {
+            if(Intent.ACTION_USER_UNLOCKED == intent?.action){
+                try {
+                    (context as FlorisApplication).unregisterReceiver(this)
+                    context.init()
+                } catch (e : Exception) {
+                    e.fillInStackTrace()
+                }
+            }
+        }
+    }
 }

app/src/main/java/dev/patrickgold/florisboard/FlorisSpellCheckerService.kt

@@ -0,0 +1,143 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard
+
+import android.service.textservice.SpellCheckerService
+import android.view.textservice.SentenceSuggestionsInfo
+import android.view.textservice.SuggestionsInfo
+import android.view.textservice.TextInfo
+import dev.patrickgold.florisboard.common.FlorisLocale
+import dev.patrickgold.florisboard.debug.LogTopic
+import dev.patrickgold.florisboard.debug.flogInfo
+import dev.patrickgold.florisboard.ime.core.Subtype
+import dev.patrickgold.florisboard.ime.core.SubtypeManager
+import dev.patrickgold.florisboard.ime.dictionary.DictionaryManager
+import dev.patrickgold.florisboard.ime.spelling.SpellingService
+import kotlinx.coroutines.runBlocking
+
+class FlorisSpellCheckerService : SpellCheckerService() {
+    companion object {
+        private const val USE_FLORIS_SUBTYPES_LOCALE: String = "zz"
+    }
+
+    private val dictionaryManager get() = DictionaryManager.default()
+    private val spellingService: SpellingService = SpellingService.globalInstance()
+    private val subtypeManager get() = SubtypeManager.default()
+
+    override fun onCreate() {
+        flogInfo(LogTopic.SPELL_EVENTS)
+
+        super.onCreate()
+        dictionaryManager.loadUserDictionariesIfNecessary()
+    }
+
+    override fun createSession(): Session {
+        flogInfo(LogTopic.SPELL_EVENTS)
+
+        return FlorisSpellCheckerSession()
+    }
+
+    override fun onDestroy() {
+        flogInfo(LogTopic.SPELL_EVENTS)
+
+        super.onDestroy()
+    }
+
+    private inner class FlorisSpellCheckerSession : Session() {
+        private var cachedSpellingLocale: FlorisLocale? = null
+
+        override fun onCreate() {
+            flogInfo(LogTopic.SPELL_EVENTS) { "Session locale: $locale" }
+
+            setupSpellingIfNecessary()
+        }
+
+        private fun setupSpellingIfNecessary() {
+            val evaluatedLocale = when (locale) {
+                null -> Subtype.DEFAULT.locale
+                USE_FLORIS_SUBTYPES_LOCALE -> (subtypeManager.getActiveSubtype() ?: Subtype.DEFAULT).locale
+                else -> FlorisLocale.from(locale)
+            }
+
+            if (evaluatedLocale != cachedSpellingLocale) {
+                cachedSpellingLocale = evaluatedLocale
+            }
+        }
+
+        private fun spellMultiple(
+            spellingLocale: FlorisLocale,
+            textInfos: Array<out TextInfo>,
+            suggestionsLimit: Int
+        ): Array<SuggestionsInfo> = runBlocking {
+            val retInfos = Array(textInfos.size) { n ->
+                val word = textInfos[n].text ?: ""
+                spellingService.spellAsync(spellingLocale, word, suggestionsLimit)
+            }
+            Array(textInfos.size) { n ->
+                retInfos[n].await().apply {
+                    setCookieAndSequence(textInfos[n].cookie, textInfos[n].sequence)
+                }
+            }
+        }
+
+        override fun onGetSuggestions(textInfo: TextInfo?, suggestionsLimit: Int): SuggestionsInfo {
+            flogInfo(LogTopic.SPELL_EVENTS) { "text=${textInfo?.text}, limit=$suggestionsLimit" }
+
+            textInfo?.text ?: return SpellingService.emptySuggestionsInfo()
+            setupSpellingIfNecessary()
+            val spellingLocale = cachedSpellingLocale ?: return SpellingService.emptySuggestionsInfo()
+
+            return spellingService.spell(spellingLocale, textInfo.text, suggestionsLimit)
+        }
+
+        override fun onGetSuggestionsMultiple(
+            textInfos: Array<out TextInfo>?,
+            suggestionsLimit: Int,
+            sequentialWords: Boolean
+        ): Array<SuggestionsInfo> {
+            flogInfo(LogTopic.SPELL_EVENTS)
+
+            textInfos ?: return emptyArray()
+            setupSpellingIfNecessary()
+            val spellingLocale = cachedSpellingLocale ?: return emptyArray()
+
+            return spellMultiple(spellingLocale, textInfos, suggestionsLimit)
+        }
+
+        override fun onGetSentenceSuggestionsMultiple(
+            textInfos: Array<out TextInfo>?,
+            suggestionsLimit: Int
+        ): Array<SentenceSuggestionsInfo> {
+            flogInfo(LogTopic.SPELL_EVENTS)
+
+            // TODO: implement custom solution here instead of calling the default implementation
+            return super.onGetSentenceSuggestionsMultiple(textInfos, suggestionsLimit)
+        }
+
+        override fun onCancel() {
+            flogInfo(LogTopic.SPELL_EVENTS)
+
+            super.onCancel()
+        }
+
+        override fun onClose() {
+            flogInfo(LogTopic.SPELL_EVENTS)
+
+            super.onClose()
+        }
+    }
+}

app/src/main/java/dev/patrickgold/florisboard/common/FlorisActivity.kt

@@ -65,12 +65,12 @@ abstract class FlorisActivity<V : ViewBinding> : AppCompatActivity(), CoroutineS
         messageSnackbar = null
     }
 
-    protected fun showMessage(@StringRes snackbarMessageResId: Int) {
+    fun showMessage(@StringRes snackbarMessageResId: Int) {
         val snackbarMessage = resources.getString(snackbarMessageResId)
         showMessage(snackbarMessage)
     }
 
-    protected fun showMessage(snackbarMessage: String) {
+    fun showMessage(snackbarMessage: String) {
         messageSnackbar?.dismiss()
         messageSnackbar = Snackbar.make(binding.root, snackbarMessage, Snackbar.LENGTH_LONG).apply {
             setAction(android.R.string.ok) {
@@ -79,39 +79,43 @@ abstract class FlorisActivity<V : ViewBinding> : AppCompatActivity(), CoroutineS
             show() }
     }
 
-    protected fun showError(throwable: Throwable) {
+    fun showError(throwable: Throwable) {
         val snackbarMessage = resources.getString(R.string.assets__error__snackbar_message)
         showError(snackbarMessage, throwable)
     }
 
-    protected fun showError(@StringRes snackbarMessageResId: Int, throwable: Throwable) {
+    fun showError(@StringRes snackbarMessageResId: Int, throwable: Throwable) {
         val snackbarMessage = resources.getString(snackbarMessageResId)
         showError(snackbarMessage, throwable)
     }
 
-    protected fun showError(snackbarMessage: String, throwable: Throwable) {
+    fun showError(snackbarMessage: String, throwable: Throwable) {
         errorDialog?.dismiss()
         errorDialog = null
         errorSnackbar?.dismiss()
         errorSnackbar = Snackbar.make(binding.root, snackbarMessage, Snackbar.LENGTH_LONG).apply {
             setAction(R.string.assets__error__details) {
-                errorDialog?.dismiss()
-                errorDialog = AlertDialog.Builder(this@FlorisActivity).run {
-                    setTitle(R.string.assets__error__details)
-                    setMessage(errorThrowable?.stackTraceToString())
-                    setPositiveButton(android.R.string.ok, null)
-                    setNeutralButton(R.string.crash_dialog__copy_to_clipboard) { _, _ ->
-                        val clipboardManager = getSystemService(Context.CLIPBOARD_SERVICE)
-                        if (clipboardManager != null && clipboardManager is ClipboardManager) {
-                            clipboardManager.setPrimaryClip(ClipData.newPlainText(errorThrowable.toString(), errorThrowable.toString()))
-                        }
-                    }
-                    create()
-                    show()
-                }
+                showErrorDialog()
             }
             show()
         }
         errorThrowable = throwable
     }
+
+    fun showErrorDialog(throwable: Throwable? = errorThrowable) {
+        errorDialog?.dismiss()
+        errorDialog = AlertDialog.Builder(this@FlorisActivity).run {
+            setTitle(R.string.assets__error__details)
+            setMessage(throwable?.stackTraceToString())
+            setPositiveButton(android.R.string.ok, null)
+            setNeutralButton(R.string.crash_dialog__copy_to_clipboard) { _, _ ->
+                val clipboardManager = getSystemService(Context.CLIPBOARD_SERVICE)
+                if (clipboardManager != null && clipboardManager is ClipboardManager) {
+                    clipboardManager.setPrimaryClip(ClipData.newPlainText(throwable.toString(), throwable.toString()))
+                }
+            }
+            create()
+            show()
+        }
+    }
 }

app/src/main/java/dev/patrickgold/florisboard/common/FlorisLocale.kt

@@ -0,0 +1,306 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.common
+
+import kotlinx.serialization.KSerializer
+import kotlinx.serialization.Serializable
+import kotlinx.serialization.descriptors.PrimitiveKind
+import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
+import kotlinx.serialization.descriptors.SerialDescriptor
+import kotlinx.serialization.encoding.Decoder
+import kotlinx.serialization.encoding.Encoder
+import java.util.*
+
+/**
+ * Project-specific locale class wrapping [java.util.Locale]. The wrapping is
+ * necessary to provide consistent language display names and tags across the
+ * whole code base.
+ *
+ * This class would be ideal for Kotlin's value classes, though AndroidX.Room
+ * does not like this at all, so this is a "normal" class.
+ *
+ * To construct a FlorisLocale, use one of the many from() methods provided.
+ *
+ * @see java.util.Locale
+ */
+@Serializable(with = FlorisLocale.Serializer::class)
+class FlorisLocale private constructor(val base: Locale) {
+    companion object {
+        /** Delimiter for a language tag. */
+        private const val DELIMITER_LANGUAGE_TAG = '-'
+        /** Delimiter for a locale tag. */
+        private const val DELIMITER_LOCALE_TAG = '_'
+
+        /** Delimiter regex to split language/locale tags. */
+        private val DELIMITER_SPLITTER = """[${DELIMITER_LANGUAGE_TAG}${DELIMITER_LOCALE_TAG}]""".toRegex()
+
+        /** Constant locale for ROOT */
+        val ROOT = from("", "", "")
+
+        /** Constant locale for ENGLISH */
+        val ENGLISH = from("en", "", "")
+
+        /**
+         * Wraps a [java.util.Locale] and returns the [FlorisLocale].
+         *
+         * @return The wrapped locale.
+         */
+        fun from(javaLocale: Locale) = FlorisLocale(javaLocale)
+
+        /**
+         * Constructs a new [FlorisLocale] with given [language].
+         *
+         * @param language A two-letter language code.
+         *
+         * @return A new [FlorisLocale].
+         */
+        fun from(language: String) = from(Locale(language))
+
+        /**
+         * Constructs a new [FlorisLocale] with given [language] and [country].
+         *
+         * @param language A two-letter language code.
+         * @param country A two-letter country code.
+         *
+         * @return A new [FlorisLocale].
+         */
+        fun from(language: String, country: String) = from(Locale(language, country))
+
+        /**
+         * Constructs a new [FlorisLocale] with given [language], [country] and [variant].
+         *
+         * @param language A two-letter language code.
+         * @param country A two-letter country code.
+         * @param variant A two-letter variant code.
+         *
+         * @return A new [FlorisLocale].
+         */
+        fun from(language: String, country: String, variant: String) = from(Locale(language, country, variant))
+
+        /**
+         * Constructs a new [FlorisLocale] from given [str].
+         *
+         * @param str Either a language or locale tag in string form.
+         *
+         * @return A new [FlorisLocale].
+         */
+        fun fromTag(str: String) = when {
+            str.contains(DELIMITER_SPLITTER) -> {
+                val lc = str.split(DELIMITER_SPLITTER)
+                if (lc.size >= 3) {
+                    from(lc[0], lc[1], lc[2])
+                } else {
+                    from(lc[0], lc[1])
+                }
+            }
+            else -> from(str)
+        }
+
+        /**
+         * Gets the current value of the default locale for this instance of
+         * the Java Virtual Machine.
+         *
+         * @see java.util.Locale.getDefault
+         */
+        fun default() = FlorisLocale(Locale.getDefault())
+    }
+
+    /**
+     * Builds a locale or language tag for this locale by using [delimiter].
+     *
+     * @param delimiter The delimiter to use between the components.
+     *
+     * @return The generated tag for this locale. May be an empty string if
+     *  [language], [country] and [variant] are not specified.
+     */
+    private fun buildLocaleString(delimiter: Char) = stringBuilder {
+        val language = base.language
+        val country = base.country
+        val variant = base.variant
+        append(language)
+        if (language.isNotBlank() && country.isNotBlank()) {
+            append(delimiter)
+        }
+        append(country)
+        if (country.isNotBlank() && variant.isNotBlank()) {
+            append(delimiter)
+        }
+        append(variant)
+    }
+
+    /**
+     * Returns the language code of this locale.
+     *
+     * @see java.util.Locale.getLanguage
+     */
+    val language: String get() = base.language
+
+    /**
+     * Returns the country/region code for this locale.
+     *
+     * @see java.util.Locale.getCountry
+     */
+    val country: String get() = base.country
+
+    /**
+     * Returns the variant code for this locale.
+     *
+     * @see java.util.Locale.getVariant
+     */
+    val variant: String get() = base.variant
+
+    /**
+     * Returns a three-letter abbreviation of this locale's language.
+     *
+     * @see java.util.Locale.getISO3Language
+     */
+    val iso3Language: String get() = base.isO3Language
+
+    /**
+     * Returns a three-letter abbreviation of this locale's country.
+     *
+     * @see java.util.Locale.getISO3Country
+     */
+    val iso3Country: String get() = base.isO3Country
+
+    /**
+     * Generates the language tag for this locale in the format `xx`,
+     * `xx-YY` or `xx-YY-zzz` and returns it as a string.
+     *
+     * xx: Two-letter language code
+     * YY: Two-letter country code
+     * zzz: Three letter variant
+     *
+     * @return The language tag for this locale. May be an empty string if
+     *  [language], [country] and [variant] are not specified.
+     */
+    fun languageTag(): String = buildLocaleString(DELIMITER_LANGUAGE_TAG)
+
+    /**
+     * Generates the locale tag for this locale in the format `xx`,
+     * `xx_YY` or `xx_YY_zzz` and returns it as a string.
+     *
+     * xx: Two-letter language code
+     * YY: Two-letter country code
+     * zzz: Three letter variant
+     *
+     * @return The locale tag for this locale. May be an empty string if
+     *  [language], [country] and [variant] are not specified.
+     */
+    fun localeTag(): String = buildLocaleString(DELIMITER_LOCALE_TAG)
+
+    /**
+     * Returns the name of this locale's language, localized to [locale].
+     *
+     * @see java.util.Locale.getDisplayLanguage
+     */
+    fun displayLanguage(locale: FlorisLocale = default()): String = base.getDisplayLanguage(locale.base)
+
+    /**
+     * Returns the name of this locale's country, localized to [locale].
+     *
+     * @see java.util.Locale.getDisplayCountry
+     */
+    fun displayCountry(locale: FlorisLocale = default()): String = base.getDisplayCountry(locale.base)
+
+    /**
+     * Returns a name for the locale's variant code that is appropriate for
+     * display to the user.
+     *
+     * @see java.util.Locale.getDisplayVariant
+     */
+    fun displayVariant(locale: FlorisLocale = default()): String = base.getDisplayVariant(locale.base)
+
+    /**
+     * Returns the display name for this locale, localized to [locale] in
+     * the format `Language`, `Language (Country)` or `Language (Country) \[VARIANT]`.
+     *
+     * @param locale The locale to use for generating the display name for
+     *  this locale, or [default] if otherwise.
+     *
+     * @return The display name for this locale. May be an empty string if
+     *  [language], [country] and [variant] are not specified.
+     */
+    fun displayName(locale: FlorisLocale = default()) = stringBuilder {
+        val languageName = displayLanguage(locale).ifBlank { base.language }
+        val countryName = displayCountry(locale).ifBlank { base.country }
+        val variantName = displayVariant(locale).ifBlank { base.variant }
+        append(languageName)
+        if (countryName.isNotBlank()) {
+            if (languageName.isNotBlank()) {
+                append(' ')
+            }
+            append('(')
+            append(countryName)
+            append(')')
+        }
+        if (variantName.isNotBlank()) {
+            if (languageName.isNotBlank() || countryName.isNotBlank()) {
+                append(' ')
+            }
+            append('[')
+            append(variantName.uppercase())
+            append(']')
+        }
+    }
+
+    /**
+     * Generate a debug string representing this locale. Not to be confused
+     * with [java.util.Locale.toString], which produces a locale tag. If such
+     * tag is needed, use [localeTag].
+     *
+     * @return The debug representation of this locale.
+     */
+    override fun toString() = "FlorisLocale { l=${base.language} c=${base.country} v=${base.variant} }"
+
+    /**
+     * Equality check for this locale.
+     */
+    override fun equals(other: Any?): Boolean {
+        if (this === other) return true
+        if (javaClass != other?.javaClass) return false
+
+        other as FlorisLocale
+
+        if (base != other.base) return false
+
+        return true
+    }
+
+    /**
+     * Returns the hash code for this locale.
+     */
+    override fun hashCode(): Int {
+        return base.hashCode()
+    }
+
+    /**
+     * The JSON (de)serializer for FlorisLocale.
+     */
+    class Serializer : KSerializer<FlorisLocale> {
+        override val descriptor: SerialDescriptor =
+            PrimitiveSerialDescriptor("FlorisLocale", PrimitiveKind.STRING)
+
+        override fun serialize(encoder: Encoder, value: FlorisLocale) {
+            encoder.encodeString(value.languageTag())
+        }
+
+        override fun deserialize(decoder: Decoder): FlorisLocale {
+            return fromTag(decoder.decodeString())
+        }
+    }
+}

app/src/main/java/dev/patrickgold/florisboard/common/Native.kt

@@ -16,6 +16,8 @@
 
 package dev.patrickgold.florisboard.common
 
+import java.nio.ByteBuffer
+
 /**
  * Type alias for a native pointer.
  */
@@ -24,7 +26,37 @@ typealias NativePtr = Long
 /**
  * Constant value for a native null pointer.
  */
-const val NATIVE_NULLPTR: NativePtr = 0
+const val NATIVE_NULLPTR: NativePtr = 0L
+
+/**
+ * Type alias for a native string in standard UTF-8 encoding.
+ */
+typealias NativeStr = ByteBuffer
+
+/**
+ * Converts a native string to a Java string.
+ */
+fun NativeStr.toJavaString(): String {
+    val bytes: ByteArray
+    if (this.hasArray()) {
+        bytes = this.array()
+    } else {
+        bytes = ByteArray(this.remaining())
+        this.get(bytes)
+    }
+    return String(bytes, Charsets.UTF_8)
+}
+
+/**
+ * Converts a Java string to a native string.
+ */
+fun String.toNativeStr(): NativeStr {
+    val bytes = this.toByteArray(Charsets.UTF_8)
+    val buffer = ByteBuffer.allocateDirect(bytes.size)
+    buffer.put(bytes)
+    buffer.rewind()
+    return buffer
+}
 
 /**
  * Generic interface for a native instance object. Defines the basic

app/src/main/java/dev/patrickgold/florisboard/common/RegexSerializer.kt

@@ -0,0 +1,36 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.common
+
+import kotlinx.serialization.KSerializer
+import kotlinx.serialization.descriptors.PrimitiveKind
+import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
+import kotlinx.serialization.descriptors.SerialDescriptor
+import kotlinx.serialization.encoding.Decoder
+import kotlinx.serialization.encoding.Encoder
+
+class RegexSerializer : KSerializer<Regex> {
+    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("ThemeValue", PrimitiveKind.STRING)
+
+    override fun serialize(encoder: Encoder, value: Regex) {
+        encoder.encodeString(value.toString())
+    }
+
+    override fun deserialize(decoder: Decoder): Regex {
+        return decoder.decodeString().toRegex()
+    }
+}

app/src/main/java/dev/patrickgold/florisboard/common/Result.kt

@@ -0,0 +1,41 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.common
+
+import kotlin.contracts.InvocationKind
+import kotlin.contracts.contract
+
+inline fun <T> resultOk(value: () -> T): Result<T> {
+    contract {
+        callsInPlace(value, InvocationKind.EXACTLY_ONCE)
+    }
+    return Result.success(value())
+}
+
+inline fun <T> resultErr(error: () -> Throwable): Result<T> {
+    contract {
+        callsInPlace(error, InvocationKind.EXACTLY_ONCE)
+    }
+    return Result.failure(error())
+}
+
+inline fun <T> resultErrStr(error: () -> String): Result<T> {
+    contract {
+        callsInPlace(error, InvocationKind.EXACTLY_ONCE)
+    }
+    return Result.failure(Exception(error()))
+}

app/src/main/java/dev/patrickgold/florisboard/common/Strings.kt

@@ -0,0 +1,35 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+@file:Suppress("NOTHING_TO_INLINE")
+
+package dev.patrickgold.florisboard.common
+
+import kotlin.contracts.InvocationKind
+import kotlin.contracts.contract
+
+inline fun <R> stringBuilder(builder: StringBuilder.() -> R): String {
+    contract {
+        callsInPlace(builder, InvocationKind.EXACTLY_ONCE)
+    }
+    val sb = StringBuilder()
+    builder(sb)
+    return sb.toString()
+}
+
+inline fun String.lowercase(locale: FlorisLocale): String = this.lowercase(locale.base)
+
+inline fun String.uppercase(locale: FlorisLocale): String = this.uppercase(locale.base)

app/src/main/java/dev/patrickgold/florisboard/common/ViewUtils.kt

@@ -20,9 +20,12 @@ package dev.patrickgold.florisboard.common
 import android.content.res.Resources
 import android.util.DisplayMetrics
 import android.view.View
+import android.view.ViewGroup
 import android.view.Window
 import android.widget.FrameLayout
 import android.widget.LinearLayout
+import androidx.core.view.children
+import androidx.core.view.isVisible
 
 /**
  * This file has been taken from the Android LatinIME project. Following modifications were done to
@@ -98,4 +101,22 @@ object ViewUtils {
     fun px2dp(px: Float): Float {
         return px / (Resources.getSystem().displayMetrics.densityDpi.toFloat() / DisplayMetrics.DENSITY_DEFAULT)
     }
+
+    fun setEnabled(view: View, v: Boolean) {
+        view.isEnabled = v
+        if (view is ViewGroup) {
+            for (childView in view.children) {
+                setEnabled(childView, v)
+            }
+        }
+    }
+
+    fun setVisible(view: View, v: Boolean) {
+        view.isVisible = v
+        if (view is ViewGroup) {
+            for (childView in view.children) {
+                setVisible(childView, v)
+            }
+        }
+    }
 }

app/src/main/java/dev/patrickgold/florisboard/debug/Flog.kt

@@ -14,15 +14,12 @@
  * limitations under the License.
  */
 
-@file:OptIn(ExperimentalContracts::class)
-
 package dev.patrickgold.florisboard.debug
 
 import android.content.Context
 import android.os.Build
 import android.util.Log
 import java.lang.ref.WeakReference
-import kotlin.contracts.ExperimentalContracts
 import kotlin.contracts.InvocationKind
 import kotlin.contracts.contract
 

app/src/main/java/dev/patrickgold/florisboard/debug/LogTopic.kt

@@ -37,8 +37,13 @@ object LogTopic {
     const val TEXT_KEYBOARD_VIEW: FlogTopic =   16u
     const val GESTURES: FlogTopic =             32u
     const val SMARTBAR: FlogTopic =             64u
+    const val THEME_MANAGER: FlogTopic =        128u
+    const val ASSET_MANAGER: FlogTopic =        256u
 
     const val GLIDE: FlogTopic =                512u
     const val CLIPBOARD: FlogTopic =            1024u
     const val CRASH_UTILITY: FlogTopic =        2048u
+
+    const val SPELL_EVENTS: FlogTopic =         4096u
+    const val EDITOR_INSTANCE: FlogTopic =      0x00_00_20_00u
 }

app/src/main/java/dev/patrickgold/florisboard/ime/clip/ClipboardHistoryItemAdapter.kt

@@ -1,6 +1,10 @@
 package dev.patrickgold.florisboard.ime.clip
 
-import android.graphics.drawable.Drawable
+import android.animation.ValueAnimator
+import android.content.ContentResolver
+import android.graphics.Bitmap
+import android.graphics.BitmapFactory
+import android.net.Uri
 import android.view.LayoutInflater
 import android.view.View
 import android.view.View.GONE
@@ -13,11 +17,16 @@ import dev.patrickgold.florisboard.R
 import dev.patrickgold.florisboard.ime.clip.provider.ClipboardItem
 import dev.patrickgold.florisboard.ime.clip.provider.ItemType
 import dev.patrickgold.florisboard.ime.core.FlorisBoard
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.MainScope
+import kotlinx.coroutines.launch
+import kotlin.math.roundToInt
 
 class ClipboardHistoryItemAdapter(
-        private val dataSet: ArrayDeque<FlorisClipboardManager.TimedClipData>,
-        private val pins: ArrayDeque<ClipboardItem>
-    ) : RecyclerView.Adapter<RecyclerView.ViewHolder>() {
+    private val dataSet: ArrayDeque<FlorisClipboardManager.TimedClipData>,
+    private val pins: ArrayDeque<ClipboardItem>
+) : RecyclerView.Adapter<RecyclerView.ViewHolder>(), CoroutineScope by MainScope() {
 
     class ClipboardHistoryTextViewHolder(view: View) : RecyclerView.ViewHolder(view) {
         val textView: TextView = view.findViewById(R.id.clipboard_history_item_text)
@@ -35,15 +44,13 @@ class ClipboardHistoryItemAdapter(
         return if (position < pins.size) {
             // is a pin
             pins[position].type.value
-        }else {
+        } else {
             // regular history item
             dataSet[position - pins.size].data.type.value
         }
     }
 
 
-
-
     override fun onCreateViewHolder(viewGroup: ViewGroup, viewType: Int): RecyclerView.ViewHolder {
         // Create a new view, which defines the UI of the list item
         val vh = when (viewType) {
@@ -53,7 +60,7 @@ class ClipboardHistoryItemAdapter(
 
                 ClipboardHistoryImageViewHolder(view)
             }
-            ItemType.TEXT.value  -> {
+            ItemType.TEXT.value -> {
                 val view = LayoutInflater.from(viewGroup.context)
                     .inflate(R.layout.clipboard_history_item_text, viewGroup, false)
 
@@ -73,7 +80,7 @@ class ClipboardHistoryItemAdapter(
                 var text = if (position < pins.size) {
                     (viewHolder.itemView as ClipboardHistoryItemView).setPinned()
                     pins[position].text
-                }else {
+                } else {
                     (viewHolder.itemView as ClipboardHistoryItemView).setUnpinned()
                     dataSet[position - pins.size].data.text
                 }
@@ -87,7 +94,7 @@ class ClipboardHistoryItemAdapter(
                 val uri = if (position < pins.size) {
                     (viewHolder.itemView as ClipboardHistoryItemView).setPinned()
                     pins[position].uri
-                }else {
+                } else {
                     (viewHolder.itemView as ClipboardHistoryItemView).setUnpinned()
                     dataSet[position - pins.size].data.uri
                 }
@@ -95,20 +102,73 @@ class ClipboardHistoryItemAdapter(
 
                 viewHolder.imgView.clipToOutline = true
                 viewHolder.imgView.visibility = GONE
-                // For very large images, this can take a bit
-                FlorisClipboardManager.getInstance().executor.execute {
-                    val resolver = FlorisBoard.getInstance().contentResolver
-                    val inputStream = resolver.openInputStream(uri!!)
 
-                    val drawable = Drawable.createFromStream(inputStream, "clipboard URI")
+                // The code looks like a mess because we're jumping across threads so much :(
+                // read dimensions (IO) -> set dimensions (UI) -> read bitmap (IO) -> set bitmap (UI)
+                launch(Dispatchers.IO) {
+                    val resolver = FlorisBoard.getInstance().contentResolver
+                    val (imgWidth, imgHeight) = getImageDimensions(resolver, uri!!)
                     viewHolder.itemView.post {
-                        viewHolder.imgView.setImageDrawable(drawable)
-                        viewHolder.imgView.visibility = VISIBLE
+                        val width = viewHolder.itemView.width
+                        val sampleSize = calcSampleSize(imgWidth, width)
+                        val params = viewHolder.itemView.layoutParams.apply {
+                            height = (width * (imgHeight.toFloat() / imgWidth)).roundToInt() + 30
+                        }
+                        viewHolder.itemView.layoutParams = params
+                        this@ClipboardHistoryItemAdapter.launch(Dispatchers.IO) {
+                            val bitmap = loadSampledBitmap(resolver, uri, sampleSize)
+                            bitmap?.let {
+                                viewHolder.itemView.post {
+                                    viewHolder.imgView.visibility = VISIBLE
+                                    val animator = ValueAnimator.ofFloat(0f, 1f)
+                                    animator.duration = 150
+                                    animator.addUpdateListener {
+                                        viewHolder.imgView.alpha = it.animatedValue as Float
+                                    }
+                                    animator.start()
+                                    viewHolder.imgView.setImageBitmap(bitmap)
+                                }
+                            }
+                        }
                     }
                 }
             }
         }
     }
+
     override fun getItemCount() = pins.size + dataSet.size
 
+    // returns (width, height)
+    private fun getImageDimensions(resolver: ContentResolver, uri: Uri): Pair<Int, Int> {
+        return BitmapFactory.Options().run {
+            inJustDecodeBounds = true
+            val stream = resolver.openInputStream(uri)
+            BitmapFactory.decodeStream(stream, null, this)
+
+            Pair(outWidth, outHeight)
+        }
+    }
+
+    private fun calcSampleSize(imgWidth: Int, reqWidth: Int): Int {
+        var inSampleSize = 2
+        while (imgWidth / inSampleSize > reqWidth) {
+            inSampleSize *= 2
+        }
+        return inSampleSize / 2
+    }
+
+    private fun loadSampledBitmap(resolver: ContentResolver, uri: Uri, inSampleSize: Int): Bitmap? {
+        return BitmapFactory.Options().run {
+            // Calculate inSampleSize
+            this.inSampleSize = inSampleSize
+
+            // Decode bitmap with inSampleSize set
+            inJustDecodeBounds = false
+            val stream2 = resolver.openInputStream(uri)
+            BitmapFactory.decodeStream(stream2, null, this)
+        }
+
+    }
+
+
 }

app/src/main/java/dev/patrickgold/florisboard/ime/clip/ClipboardHistoryView.kt

@@ -64,7 +64,7 @@ class ClipboardHistoryView : LinearLayout, FlorisBoard.EventListener,
     }
 
     override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
-        val height = florisboard?.inputView?.desiredMediaKeyboardViewHeight ?: 0.0f
+        val height = florisboard?.uiBinding?.inputView?.desiredMediaKeyboardViewHeight ?: 0.0f
         super.onMeasure(widthMeasureSpec, MeasureSpec.makeMeasureSpec(height.roundToInt(), MeasureSpec.EXACTLY))
     }
 

app/src/main/java/dev/patrickgold/florisboard/ime/clip/ClipboardInputManager.kt

@@ -3,17 +3,16 @@ package dev.patrickgold.florisboard.ime.clip
 import android.annotation.SuppressLint
 import android.view.MotionEvent
 import android.view.View
-import android.widget.*
 import androidx.recyclerview.widget.RecyclerView
 import androidx.recyclerview.widget.StaggeredGridLayoutManager
 import dev.patrickgold.florisboard.BuildConfig
 import dev.patrickgold.florisboard.R
+import dev.patrickgold.florisboard.databinding.FlorisboardBinding
 import dev.patrickgold.florisboard.ime.clip.provider.ClipboardItem
 import dev.patrickgold.florisboard.ime.core.FlorisBoard
 import dev.patrickgold.florisboard.ime.core.InputKeyEvent
-import dev.patrickgold.florisboard.ime.core.InputView
 import dev.patrickgold.florisboard.ime.text.key.KeyCode
-import dev.patrickgold.florisboard.ime.text.keyboard.BasicTextKeyData
+import dev.patrickgold.florisboard.ime.text.keyboard.TextKeyData
 import kotlinx.coroutines.*
 import kotlin.math.pow
 
@@ -43,27 +42,23 @@ class ClipboardInputManager private constructor() : CoroutineScope by MainScope(
         florisboard.addEventListener(this)
     }
 
-    /**
-     * Called when a new input view has been registered. Used to initialize all media-relevant
-     * views and layouts.
-     */
     @SuppressLint("ClickableViewAccessibility")
-    override fun onRegisterInputView(inputView: InputView) {
-        inputView.findViewById<ImageButton>(R.id.back_to_keyboard_button)
+    override fun onInitializeInputUi(uiBinding: FlorisboardBinding) {
+        uiBinding.clipboard.backToKeyboardButton
+            .setOnTouchListener { view, event -> onButtonPressEvent(view, event) }
+        uiBinding.clipboard.clearClipboardHistory
             .setOnTouchListener { view, event -> onButtonPressEvent(view, event) }
 
-        inputView.findViewById<ImageButton>(R.id.clear_clipboard_history)
-            .setOnTouchListener { view, event -> onButtonPressEvent(view, event) }
+        recyclerView = uiBinding.clipboard.clipboardHistoryItems.also {
+            if (BuildConfig.DEBUG && adapter == null) {
+                error("initClipboard() not called")
+            }
 
-        recyclerView = inputView.findViewById(R.id.clipboard_history_items)
-
-        if (BuildConfig.DEBUG && adapter == null) {
-            error("initClipboard() not called")
+            it.adapter = adapter
+            val manager = StaggeredGridLayoutManager(2, StaggeredGridLayoutManager.VERTICAL)
+            it.layoutManager = manager
         }
 
-        recyclerView!!.adapter = adapter
-        val manager = StaggeredGridLayoutManager(2, StaggeredGridLayoutManager.VERTICAL)
-        recyclerView!!.layoutManager = manager
     }
 
     /**
@@ -78,7 +73,7 @@ class ClipboardInputManager private constructor() : CoroutineScope by MainScope(
      * Returns a reference to the [ClipboardHistoryView]
      */
     fun getClipboardHistoryView(): ClipboardHistoryView {
-        return FlorisBoard.getInstance().inputView?.mainViewFlipper?.getChildAt(2) as ClipboardHistoryView
+        return florisboard.uiBinding?.mainViewFlipper?.getChildAt(2) as ClipboardHistoryView
     }
 
     /**
@@ -133,15 +128,14 @@ class ClipboardInputManager private constructor() : CoroutineScope by MainScope(
         event ?: return false
 
         val data = when (view.id) {
-            R.id.back_to_keyboard_button -> BasicTextKeyData(code = KeyCode.SWITCH_TO_TEXT_CONTEXT)
-            R.id.clear_clipboard_history -> BasicTextKeyData(code = KeyCode.CLEAR_CLIPBOARD_HISTORY)
+            R.id.back_to_keyboard_button -> TextKeyData(code = KeyCode.SWITCH_TO_TEXT_CONTEXT)
+            R.id.clear_clipboard_history -> TextKeyData(code = KeyCode.CLEAR_CLIPBOARD_HISTORY)
             else -> null
         } ?: return false
 
         when (event.actionMasked) {
             MotionEvent.ACTION_DOWN -> {
-                florisboard.keyPressVibrate()
-                florisboard.keyPressSound(data)
+                florisboard.inputFeedbackManager.keyPress(data)
                 florisboard.textInputManager.inputEventDispatcher.send(InputKeyEvent.down(data))
             }
             MotionEvent.ACTION_UP -> {

app/src/main/java/dev/patrickgold/florisboard/ime/clip/FlorisClipboardManager.kt

@@ -3,17 +3,17 @@ package dev.patrickgold.florisboard.ime.clip
 import android.content.ClipboardManager
 import android.content.Context
 import android.content.Context.CLIPBOARD_SERVICE
-import android.os.Handler
-import android.os.Looper
+import dev.patrickgold.florisboard.debug.flogDebug
 import dev.patrickgold.florisboard.ime.clip.provider.*
 import dev.patrickgold.florisboard.ime.core.FlorisBoard
 import dev.patrickgold.florisboard.ime.core.Preferences
-import dev.patrickgold.florisboard.util.cancelAll
-import dev.patrickgold.florisboard.util.postAtScheduledRate
-import timber.log.Timber
+import kotlinx.coroutines.CoroutineScope
+import kotlinx.coroutines.Dispatchers
+import kotlinx.coroutines.Job
+import kotlinx.coroutines.MainScope
+import kotlinx.coroutines.delay
+import kotlinx.coroutines.launch
 import java.io.Closeable
-import java.util.*
-import java.util.concurrent.ExecutorService
 import kotlin.collections.ArrayDeque
 
 /**
@@ -37,9 +37,9 @@ import kotlin.collections.ArrayDeque
  *
  * [ClipboardPopupView] is the view representing a popup displayed when long pressing on a clipboard history item.
  */
-class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryClipChangedListener, Closeable {
+class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryClipChangedListener, Closeable,
+    CoroutineScope by MainScope() {
     private lateinit var pinsDao: PinnedClipboardItemDao
-    lateinit var executor: ExecutorService
 
     // Using ArrayDeque because it's "technically" the correct data structure (I think).
     // Newest stored first, oldest stored last.
@@ -48,8 +48,8 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
     private var current: ClipboardItem? = null
     private var onPrimaryClipChangedListeners: ArrayList<OnPrimaryClipChangedListener> = arrayListOf()
     private lateinit var systemClipboardManager: ClipboardManager
-    private lateinit var handler: Handler
     private val prefs get() = Preferences.default()
+    private lateinit var cleanUpJob: Job
 
     data class TimedClipData(val data: ClipboardItem, val timeUTC: Long)
 
@@ -108,21 +108,44 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
         val clipboardPrefs = prefs.clipboard
 
         if (clipboardPrefs.enableHistory) {
-            if (clipboardPrefs.limitHistorySize) {
-                var numRemoved = 0
-                while (history.size >= clipboardPrefs.maxHistorySize) {
-                    numRemoved += 1
-                    history.removeLast().data.close()
-                }
-                ClipboardInputManager.getInstance().notifyItemRangeRemoved(history.size, numRemoved)
-            }
+            val clipboardInputManager = ClipboardInputManager.getInstance()
 
-            val timed = TimedClipData(newData, System.currentTimeMillis())
-            history.addFirst(timed)
-            ClipboardInputManager.getInstance().notifyItemInserted(pins.size)
+            val historyElement = history.firstOrNull { it.data.type == ItemType.TEXT && it.data.text == newData.text }
+            if (historyElement != null) {
+                moveToTheBeginning(historyElement, newData, clipboardInputManager)
+            } else {
+                if (clipboardPrefs.limitHistorySize) {
+                    var numRemoved = 0
+                    while (history.size >= clipboardPrefs.maxHistorySize) {
+                        numRemoved += 1
+                        history.removeLast().data.close()
+                    }
+                    clipboardInputManager.notifyItemRangeRemoved(history.size, numRemoved)
+                }
+
+                createAndAddNewTimedClipData(newData)
+                clipboardInputManager.notifyItemInserted(pins.size)
+            }
         }
     }
 
+    /**
+     * Moves a ClipboardItem to the beginning of the history by removing the old one and creating a new one
+     */
+    private fun moveToTheBeginning(
+        historyElement: TimedClipData,
+        newData: ClipboardItem,
+        clipboardInputManager: ClipboardInputManager
+    ) {
+        val elementsPosition = history.indexOf(historyElement)
+        history.remove(historyElement)
+
+        createAndAddNewTimedClipData(newData)
+
+        clipboardInputManager.notifyItemMoved(elementsPosition, 0)
+        clipboardInputManager.notifyItemChanged(0)
+    }
+
     /**
      * Used so that [onPrimaryClipChanged] knows whether it was called by [changeCurrent] (and hence shouldn't update
      * history)
@@ -195,7 +218,8 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
         val systemPrimaryClip = systemClipboardManager.primaryClip
 
         if (systemPrimaryClip?.getItemAt(0)?.text == null &&
-            systemPrimaryClip?.getItemAt(0)?.uri == null) {
+            systemPrimaryClip?.getItemAt(0)?.uri == null
+        ) {
             return
         }
 
@@ -213,7 +237,7 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
         } else if (prefs.clipboard.enableHistory) {
             // in the event history is enabled, and it should be updated it is updated
             if (shouldUpdateHistory) {
-                updateHistory(ClipboardItem.fromClipData(systemPrimaryClip, false))
+                updateHistory(ClipboardItem.fromClipData(systemPrimaryClip, true))
             } else {
                 shouldUpdateHistory = true
             }
@@ -231,7 +255,7 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
      */
     override fun close() {
         systemClipboardManager.removePrimaryClipChangedListener(this)
-        handler.cancelAll()
+        cleanUpJob.cancel()
         instance = null
     }
 
@@ -244,38 +268,48 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
      * @param context Required to register as an onPrimaryClipChangedListener of ClipboardManager
      */
     fun initialize(context: Context) {
-        systemClipboardManager = (context.getSystemService(CLIPBOARD_SERVICE) as ClipboardManager)
-        systemClipboardManager.addPrimaryClipChangedListener(this)
+        try {
+            systemClipboardManager = (context.getSystemService(CLIPBOARD_SERVICE) as ClipboardManager)
+            systemClipboardManager.addPrimaryClipChangedListener(this)
 
-        val cleanUpClipboard = Runnable {
-            if (!prefs.clipboard.cleanUpOld) {
-                return@Runnable
+            val cleanUpClipboard = Runnable {
+                if (!prefs.clipboard.cleanUpOld) {
+                    return@Runnable
+                }
+
+                val currentTime = System.currentTimeMillis()
+                var numToPop = 0
+                val expiryTime = prefs.clipboard.cleanUpAfter * 60 * 1000
+                for (item in history.asReversed()) {
+                    if (item.timeUTC + expiryTime < currentTime) {
+                        numToPop += 1
+                    } else {
+                        break
+                    }
+                }
+                for (i in 0 until numToPop) {
+                    history.removeLast().data.close()
+                }
+                ClipboardInputManager.getInstance().notifyItemRangeRemoved(pins.size + history.size, numToPop)
             }
-
-            val currentTime = System.currentTimeMillis()
-            var numToPop = 0
-            val expiryTime = prefs.clipboard.cleanUpAfter * 60 * 1000
-            for (item in history.asReversed()) {
-                if (item.timeUTC + expiryTime < currentTime) {
-                    numToPop += 1
-                } else {
-                    break
+            FlorisBoard.getInstance().clipInputManager.initClipboard(this.history, this.pins)
+            cleanUpJob = launch(Dispatchers.Main) {
+                while (true) {
+                    cleanUpClipboard.run()
+                    delay(INTERVAL)
                 }
             }
-            for (i in 0 until numToPop) {
-                history.removeLast().data.close()
+            launch(Dispatchers.IO) {
+                pinsDao = PinnedItemsDatabase.getInstance().clipboardItemDao()
+                pinsDao.getAll().toCollection(pins)
+                try {
+                    FlorisContentProvider.getInstance().initIfNotAlready()
+                } catch (e: Exception) {
+                    e.fillInStackTrace()
+                }
             }
-            ClipboardInputManager.getInstance().notifyItemRangeRemoved(pins.size + history.size, numToPop)
-        }
-        FlorisBoard.getInstance().clipInputManager.initClipboard(this.history, this.pins)
-        handler = Handler(Looper.getMainLooper())
-        prefs
-        handler.postAtScheduledRate(0, INTERVAL, cleanUpClipboard)
-        executor = FlorisBoard.getInstance().asyncExecutor
-        executor.execute {
-            pinsDao = PinnedItemsDatabase.getInstance().clipboardItemDao()
-            pinsDao.getAll().toCollection(this.pins)
-            FlorisContentProvider.getInstance().initIfNotAlready()
+        } catch (e : Exception) {
+            e.fillInStackTrace()
         }
     }
 
@@ -284,16 +318,17 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
      */
     fun clearHistoryWithAnimation() {
         val clipInputManager = FlorisBoard.getInstance().clipInputManager
-        val delay = clipInputManager.clearClipboardWithAnimation(pins.size, history.size)
+        val animationDelay = clipInputManager.clearClipboardWithAnimation(pins.size, history.size)
 
-        handler.postDelayed({
+        launch(Dispatchers.Main) {
+            delay(animationDelay)
             val size = history.size
             for (item in history) {
                 item.data.close()
             }
             history.clear()
             clipInputManager.notifyItemRangeRemoved(pins.size, size)
-        }, delay)
+        }
     }
 
     fun pinClip(adapterPos: Int) {
@@ -303,7 +338,7 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
         clipInputManager.notifyItemMoved(adapterPos, 0)
         clipInputManager.notifyItemChanged(0)
 
-        executor.execute {
+        launch(Dispatchers.IO) {
             val uid = pinsDao.insert(pin.data)
             pin.data.uid = uid
         }
@@ -341,23 +376,29 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
             ClipboardInputManager.getInstance().notifyItemRangeRemoved(history.size, numRemoved)
         }
 
-        val timed = TimedClipData(item, System.currentTimeMillis())
-        history.addFirst(timed)
+        createAndAddNewTimedClipData(item)
 
         clipInputManager.notifyItemMoved(adapterPos, pins.size)
         clipInputManager.notifyItemChanged(pins.size)
 
-        executor.execute {
+        launch(Dispatchers.IO) {
             pinsDao.delete(item)
         }
     }
 
+    /**
+     * Creates a new TimedClipData and adds it to the history
+     */
+    private fun createAndAddNewTimedClipData(newData: ClipboardItem) {
+        val timed = TimedClipData(newData, System.currentTimeMillis())
+        history.addFirst(timed)
+    }
+
     fun removeClip(pos: Int) {
         when {
             pos < pins.size -> {
                 val item = pins.removeAt(pos)
-                executor.execute {
-                    Timber.d("removing pin")
+                launch(Dispatchers.IO) {
                     pinsDao.delete(item)
                 }
                 item.close()
@@ -386,7 +427,9 @@ class FlorisClipboardManager private constructor() : ClipboardManager.OnPrimaryC
             clipItem.mimeTypes.any { clipType ->
                 if (editorType != null) {
                     compareMimeTypes(clipType, editorType)
-                }else { false }
+                } else {
+                    false
+                }
             }
         } == true
     }

app/src/main/java/dev/patrickgold/florisboard/ime/clip/provider/FlorisDatabase.kt

@@ -22,7 +22,6 @@ enum class ItemType(val value: Int) {
 
 /**
  * Represents an item on the clipboard.
- * The URI stored belongs to FlorisContentProvider, not whatever app copied the image
  *
  * If type == ItemType.IMAGE there must be a uri set
  * if type == ItemType.TEXT there must be a text set

app/src/main/java/dev/patrickgold/florisboard/ime/core/FlorisBoard.kt

@@ -23,26 +23,22 @@ import android.content.res.Configuration
 import android.graphics.Color
 import android.inputmethodservice.ExtractEditText
 import android.inputmethodservice.InputMethodService
-import android.media.AudioManager
 import android.os.Build
 import android.os.Bundle
 import android.os.Handler
 import android.os.Looper
-import android.os.VibrationEffect
-import android.os.Vibrator
 import android.util.Size
 import android.view.ContextThemeWrapper
 import android.view.Gravity
-import android.view.HapticFeedbackConstants
 import android.view.LayoutInflater
 import android.view.View
 import android.view.ViewGroup
 import android.view.ViewTreeObserver
 import android.view.WindowManager
 import android.view.inputmethod.EditorInfo
+import android.view.inputmethod.ExtractedText
 import android.view.inputmethod.InlineSuggestionsRequest
 import android.view.inputmethod.InlineSuggestionsResponse
-import android.view.inputmethod.InputConnection
 import android.view.inputmethod.InputMethodManager
 import android.widget.Button
 import android.widget.FrameLayout
@@ -57,7 +53,6 @@ import dev.patrickgold.florisboard.crashutility.CrashUtility
 import dev.patrickgold.florisboard.debug.*
 import dev.patrickgold.florisboard.ime.clip.ClipboardInputManager
 import dev.patrickgold.florisboard.ime.clip.FlorisClipboardManager
-import dev.patrickgold.florisboard.ime.keyboard.KeyData
 import dev.patrickgold.florisboard.ime.landscapeinput.LandscapeInputUiMode
 import dev.patrickgold.florisboard.ime.media.MediaInputManager
 import dev.patrickgold.florisboard.ime.onehanded.OneHandedMode
@@ -67,14 +62,14 @@ import dev.patrickgold.florisboard.ime.text.composing.Appender
 import dev.patrickgold.florisboard.ime.text.composing.Composer
 import dev.patrickgold.florisboard.ime.text.gestures.SwipeAction
 import dev.patrickgold.florisboard.ime.text.key.CurrencySet
-import dev.patrickgold.florisboard.ime.text.key.KeyCode
-import dev.patrickgold.florisboard.ime.text.keyboard.KeyboardMode
 import dev.patrickgold.florisboard.ime.text.keyboard.TextKeyData
 import dev.patrickgold.florisboard.ime.theme.Theme
 import dev.patrickgold.florisboard.ime.theme.ThemeManager
 import dev.patrickgold.florisboard.setup.SetupActivity
 import dev.patrickgold.florisboard.util.AppVersionUtils
 import dev.patrickgold.florisboard.common.ViewUtils
+import dev.patrickgold.florisboard.databinding.FlorisboardBinding
+import dev.patrickgold.florisboard.ime.keyboard.InputFeedbackManager
 import dev.patrickgold.florisboard.ime.keyboard.KeyboardState
 import dev.patrickgold.florisboard.ime.keyboard.updateKeyboardState
 import dev.patrickgold.florisboard.util.debugSummarize
@@ -91,8 +86,6 @@ import kotlinx.serialization.Serializable
 import kotlinx.serialization.Transient
 import java.lang.ref.WeakReference
 import java.util.concurrent.CopyOnWriteArrayList
-import java.util.concurrent.ExecutorService
-import java.util.concurrent.Executors
 
 /**
  * Variable which holds the current [FlorisBoard] instance. To get this instance from another
@@ -128,19 +121,17 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     private val prefs: Preferences get() = Preferences.default()
     val activeState: KeyboardState = KeyboardState.new()
 
-    private var extractEditLayout: WeakReference<ViewGroup?> = WeakReference(null)
-    var inputView: InputView? = null
+    var uiBinding: FlorisboardBinding? = null
         private set
-    private var inputWindowView: InputWindowView? = null
+    private var extractEditLayout: WeakReference<ViewGroup?> = WeakReference(null)
     var popupLayerView: PopupLayerView? = null
         private set
     private var eventListeners: CopyOnWriteArrayList<EventListener> = CopyOnWriteArrayList()
 
-    private var audioManager: AudioManager? = null
     var imeManager: InputMethodManager? = null
+    lateinit var inputFeedbackManager: InputFeedbackManager
     var florisClipboardManager: FlorisClipboardManager? = null
     private val themeManager: ThemeManager = ThemeManager.default()
-    private var vibrator: Vibrator? = null
 
     private var internalBatchNestingLevel: Int = 0
     private val internalSelectionCache = object {
@@ -153,14 +144,13 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         var candidatesEnd: Int = -1
     }
 
-    var activeEditorInstance: EditorInstance = EditorInstance.default()
+    lateinit var activeEditorInstance: EditorInstance
 
     val subtypeManager: SubtypeManager get() = SubtypeManager.default()
     val composer: Composer get() = subtypeManager.imeConfig.composerFromName.getValue(activeSubtype.composerName)
     lateinit var activeSubtype: Subtype
     private var currentThemeIsNight: Boolean = false
     private var currentThemeResId: Int = 0
-    private var isNumberRowVisible: Boolean = false
     private var isWindowShown: Boolean = false
 
     private var responseState = ResponseState.RESET
@@ -181,15 +171,11 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
             textInputManager = TextInputManager.getInstance()
             mediaInputManager = MediaInputManager.getInstance()
             clipInputManager = ClipboardInputManager.getInstance()
-
-            System.loadLibrary("florisboard-native")
         } catch (e: Exception) {
             CrashUtility.stageException(e)
         }
     }
 
-    lateinit var asyncExecutor: ExecutorService
-
     companion object {
         @Synchronized
         fun getInstance(): FlorisBoard {
@@ -227,21 +213,19 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
                 flogInfo(LogTopic.IMS_EVENTS)
                 serviceLifecycleDispatcher.onServicePreSuperOnCreate()
 
+                activeEditorInstance = EditorInstance(this, activeState)
+
                 imeManager = getSystemService(Context.INPUT_METHOD_SERVICE) as? InputMethodManager
-                audioManager = getSystemService(Context.AUDIO_SERVICE) as? AudioManager
-                vibrator = getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator
-                prefs.sync()
+                inputFeedbackManager = InputFeedbackManager.new(this)
                 activeSubtype = subtypeManager.getActiveSubtype() ?: Subtype.DEFAULT
 
                 currentThemeIsNight = themeManager.activeTheme.isNightTheme
                 currentThemeResId = getDayNightBaseThemeId(currentThemeIsNight)
-                isNumberRowVisible = prefs.keyboard.numberRow
                 setTheme(currentThemeResId)
                 themeManager.registerOnThemeUpdatedListener(this)
 
                 AppVersionUtils.updateVersionOnInstallAndLastUse(this, prefs)
 
-                asyncExecutor = Executors.newSingleThreadExecutor()
                 florisClipboardManager = FlorisClipboardManager.getInstance().also {
                     it.initialize(this)
                     it.addPrimaryClipChangedListener(this)
@@ -267,13 +251,28 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         updateThemeContext(currentThemeResId)
 
         popupLayerView = PopupLayerView(themeContext)
+        window?.window?.findViewById<View>(android.R.id.content)?.let { content ->
+            if (content is ViewGroup) {
+                content.addView(popupLayerView)
+            }
+        }
 
-        inputWindowView = LayoutInflater.from(themeContext).inflate(R.layout.florisboard, null) as? InputWindowView
-        inputWindowView?.isHapticFeedbackEnabled = true
+        uiBinding = FlorisboardBinding.inflate(LayoutInflater.from(themeContext))
 
-        eventListeners.toList().forEach { it?.onCreateInputView() }
+        eventListeners.toList().forEach { it?.onInitializeInputUi(uiBinding!!) }
 
-        return inputWindowView
+        return uiBinding!!.inputWindowView
+    }
+
+    fun initWindow() {
+        flogInfo(LogTopic.IMS_EVENTS)
+
+        updateSoftInputWindowLayoutParameters()
+        updateOneHandedPanelVisibility()
+
+        themeManager.requestThemeUpdate(this)
+
+        dispatchCurrentStateToInputUi()
     }
 
     /**
@@ -315,17 +314,15 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
             it.close()
             florisClipboardManager = null
         }
-        audioManager = null
         imeManager = null
-        vibrator = null
         popupLayerView = null
-        inputView = null
-        inputWindowView = null
-        florisboardInstance = null
+        uiBinding = null
 
         eventListeners.toList().forEach { it?.onDestroy() }
         eventListeners.clear()
         super.onDestroy()
+
+        florisboardInstance = null
     }
 
     override fun onEvaluateFullscreenMode(): Boolean {
@@ -347,6 +344,15 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         updateSoftInputWindowLayoutParameters()
     }
 
+    override fun onUpdateExtractedText(token: Int, text: ExtractedText?) {
+        super.onUpdateExtractedText(token, text)
+        activeEditorInstance.updateText(token, text)
+    }
+
+    override fun onUpdateExtractingViews(ei: EditorInfo?) {
+        super.onUpdateExtractingViews(ei)
+    }
+
     override fun onUpdateExtractingVisibility(ei: EditorInfo?) {
         isExtractViewShown = activeState.isRichInputEditor && when (prefs.keyboard.landscapeInputUiMode) {
             LandscapeInputUiMode.DYNAMICALLY_SHOW -> !activeState.imeOptions.flagNoExtractUi
@@ -355,22 +361,8 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         }
     }
 
-    fun registerInputView(inputView: InputView) {
-        flogInfo(LogTopic.IMS_EVENTS)
-
-        window?.window?.findViewById<View>(android.R.id.content)?.let { content ->
-            if (content is ViewGroup) {
-                popupLayerView?.let { content.addView(it) }
-            }
-        }
-        this.inputView = inputView
-        updateSoftInputWindowLayoutParameters()
-        updateOneHandedPanelVisibility()
-        themeManager.notifyCallbackReceivers()
-        setActiveInput(R.id.text_input)
-        dispatchCurrentStateToInputUi()
-
-        eventListeners.toList().forEach { it?.onRegisterInputView(inputView) }
+    override fun onBindInput() {
+        activeEditorInstance.bindInput()
     }
 
     override fun onStartInput(attribute: EditorInfo?, restarting: Boolean) {
@@ -382,7 +374,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         } else {
             ResponseState.RESET
         }
-        currentInputConnection?.requestCursorUpdates(InputConnection.CURSOR_UPDATE_MONITOR)
+        activeEditorInstance.startInput(attribute)
     }
 
     override fun onStartInputView(info: EditorInfo?, restarting: Boolean) {
@@ -392,9 +384,10 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         super.onStartInputView(info, restarting)
         if (info != null) {
             activeState.update(info)
+            activeState.isSelectionMode = (info.initialSelEnd - info.initialSelStart) != 0
         }
-        activeEditorInstance = EditorInstance.from(info, this, activeState)
-        themeManager.updateRemoteColorValues(activeEditorInstance.packageName)
+        activeEditorInstance.startInputView(info)
+        themeManager.updateRemoteColorValues(activeEditorInstance.packageName ?: "")
         eventListeners.toList().forEach {
             it?.onStartInputView(activeEditorInstance, restarting)
         }
@@ -404,13 +397,12 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     override fun onFinishInputView(finishingInput: Boolean) {
         flogInfo(LogTopic.IMS_EVENTS) { "finishingInput=$finishingInput" }
 
-        if (finishingInput) {
-            activeEditorInstance = EditorInstance.default()
-        } else {
+        if (!finishingInput) {
             if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
                 textInputManager.smartbarView?.clearInlineSuggestions()
             }
         }
+        activeEditorInstance.finishInputView()
 
         super.onFinishInputView(finishingInput)
         eventListeners.toList().forEach { it?.onFinishInputView(finishingInput) }
@@ -420,10 +412,14 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     override fun onFinishInput() {
         flogInfo(LogTopic.IMS_EVENTS)
 
-        currentInputConnection?.requestCursorUpdates(0)
+        activeEditorInstance.finishInput()
         super.onFinishInput()
     }
 
+    override fun onUnbindInput() {
+        activeEditorInstance.unbindInput()
+    }
+
     @RequiresApi(Build.VERSION_CODES.R)
     override fun onCreateInlineSuggestionsRequest(uiExtras: Bundle): InlineSuggestionsRequest? {
         return if (prefs.smartbar.enabled && prefs.suggestion.api30InlineSuggestionsEnabled) {
@@ -433,12 +429,12 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
             val stylesBundle = themeManager.createInlineSuggestionUiStyleBundle(themeContext)
             InlinePresentationSpec.Builder(
                 Size(
-                    inputView?.desiredInlineSuggestionsMinWidth ?: 0,
-                    inputView?.desiredInlineSuggestionsMinHeight ?: 0
+                    uiBinding?.inputView?.desiredInlineSuggestionsMinWidth ?: 0,
+                    uiBinding?.inputView?.desiredInlineSuggestionsMinHeight ?: 0
                 ),
                 Size(
-                    inputView?.desiredInlineSuggestionsMaxWidth ?: 0,
-                    inputView?.desiredInlineSuggestionsMaxHeight ?: 0
+                    uiBinding?.inputView?.desiredInlineSuggestionsMaxWidth ?: 0,
+                    uiBinding?.inputView?.desiredInlineSuggestionsMaxHeight ?: 0
                 )
             ).let { spec ->
                 spec.setStyle(stylesBundle)
@@ -490,7 +486,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     }
 
     fun dispatchCurrentStateToInputUi() {
-        inputView?.updateKeyboardState(activeState)
+        uiBinding?.inputView?.updateKeyboardState(activeState)
     }
 
     override fun onWindowShown() {
@@ -503,24 +499,23 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         }
         isWindowShown = true
 
-        prefs.sync()
-        val newIsNumberRowVisible = prefs.keyboard.numberRow
-        if (isNumberRowVisible != newIsNumberRowVisible) {
-            textInputManager.keyboards.clear(KeyboardMode.CHARACTERS)
-            isNumberRowVisible = newIsNumberRowVisible
+        val newActiveSubtype = subtypeManager.getActiveSubtype() ?: Subtype.DEFAULT
+        if (newActiveSubtype != activeSubtype) {
+            activeSubtype = newActiveSubtype
+            onSubtypeChanged(activeSubtype, true)
+        } else {
+            onSubtypeChanged(activeSubtype, false)
         }
-        themeManager.update()
-        activeSubtype = subtypeManager.getActiveSubtype() ?: Subtype.DEFAULT
-        onSubtypeChanged(activeSubtype)
         setActiveInput(R.id.text_input)
         updateOneHandedPanelVisibility()
+        themeManager.update()
 
         if (prefs.devtools.enabled && prefs.devtools.showHeapMemoryStats) {
             devtoolsOverlaySyncJob?.cancel()
             devtoolsOverlaySyncJob = uiScope.launch(Dispatchers.Default) {
                 while (true) {
                     if (!isActive) break
-                    withContext(Dispatchers.Main) { inputView?.invalidate() }
+                    withContext(Dispatchers.Main) { uiBinding?.inputView?.invalidate() }
                     delay(1000)
                 }
             }
@@ -606,13 +601,12 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         activeState.isSelectionMode = (newSelEnd - newSelStart) != 0
         if (internalBatchNestingLevel == 0) {
             flogInfo(LogTopic.IMS_EVENTS) { "onUpdateSelection($oldSelStart, $oldSelEnd, $newSelStart, $newSelEnd, $candidatesStart, $candidatesEnd)" }
-            activeEditorInstance.onUpdateSelection(
+            activeEditorInstance.updateSelection(
                 oldSelStart, oldSelEnd,
                 newSelStart, newSelEnd,
                 candidatesStart, candidatesEnd
             )
             eventListeners.toList().forEach { it?.onUpdateSelection() }
-            dispatchCurrentStateToInputUi()
         } else {
             flogInfo(LogTopic.IMS_EVENTS) { "onUpdateSelection($oldSelStart, $oldSelEnd, $newSelStart, $newSelEnd, $candidatesStart, $candidatesEnd): caught due to internal batch level of $internalBatchNestingLevel!" }
             if (internalSelectionCache.selectionCatchCount++ == 0) {
@@ -624,6 +618,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
             internalSelectionCache.candidatesStart = candidatesStart
             internalSelectionCache.candidatesEnd = candidatesEnd
         }
+        dispatchCurrentStateToInputUi()
     }
 
     override fun onThemeUpdated(theme: Theme) {
@@ -664,8 +659,8 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         w.decorView.systemUiVisibility = flags
 
         // Update InputView theme
-        inputView?.setBackgroundColor(theme.getAttr(Theme.Attr.KEYBOARD_BACKGROUND).toSolidColor().color)
-        inputView?.invalidate()
+        uiBinding?.inputView?.setBackgroundColor(theme.getAttr(Theme.Attr.KEYBOARD_BACKGROUND).toSolidColor().color)
+        uiBinding?.inputView?.invalidate()
 
         // Update ExtractTextView theme and attributes
         extractEditLayout.get()?.let { eel ->
@@ -699,8 +694,8 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
 
     override fun onComputeInsets(outInsets: Insets?) {
         super.onComputeInsets(outInsets)
-        val inputView = this.inputView ?: return
-        val inputWindowView = this.inputWindowView ?: return
+        val inputView = uiBinding?.inputView ?: return
+        val inputWindowView = uiBinding?.inputWindowView ?: return
         // TODO: Check also if the keyboard is currently suppressed by a hardware keyboard
 
         if (!isInputViewShown) {
@@ -724,7 +719,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     private fun updateSoftInputWindowLayoutParameters() {
         val w = window?.window ?: return
         ViewUtils.updateLayoutHeightOf(w, WindowManager.LayoutParams.MATCH_PARENT)
-        val inputWindowView = this.inputWindowView
+        val inputWindowView = uiBinding?.inputWindowView
         if (inputWindowView != null) {
             val layoutHeight = if (isFullscreenMode) {
                 WindowManager.LayoutParams.WRAP_CONTENT
@@ -738,84 +733,6 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
         }
     }
 
-    /**
-     * Makes a key press vibration if the user has this feature enabled in the preferences.
-     */
-    fun keyPressVibrate(isMovingGestureEffect: Boolean = false) {
-        if (prefs.keyboard.vibrationEnabled) {
-            var vibrationDuration = prefs.keyboard.vibrationDuration.toLong()
-            var vibrationStrength = prefs.keyboard.vibrationStrength
-
-            if (!prefs.keyboard.vibrationEnabledSystem && vibrationDuration < 0 && vibrationStrength < 0) {
-                return
-            }
-
-            val hapticsPerformed = if (vibrationDuration < 0 && vibrationStrength < 0) {
-                if (isMovingGestureEffect && Build.VERSION.SDK_INT >= Build.VERSION_CODES.O_MR1) {
-                    inputWindowView?.performHapticFeedback(HapticFeedbackConstants.TEXT_HANDLE_MOVE)
-                } else {
-                    inputWindowView?.performHapticFeedback(HapticFeedbackConstants.KEYBOARD_TAP)
-                }
-            } else {
-                false
-            }
-
-            if (hapticsPerformed == true) {
-                return
-            }
-
-            if (vibrationDuration == -1L) {
-                vibrationDuration = 36
-            }
-            if (isMovingGestureEffect) {
-                vibrationDuration = (vibrationDuration / 8.0).toLong().coerceAtLeast(1)
-            }
-            if (vibrationStrength == -1 && Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
-                vibrationStrength = VibrationEffect.DEFAULT_AMPLITUDE
-            } else if (vibrationStrength == -1) {
-                vibrationStrength = 36
-            }
-            if (isMovingGestureEffect && vibrationStrength > 0) {
-                vibrationStrength = (vibrationStrength / 2.0).toInt().coerceAtLeast(1)
-            } else if (isMovingGestureEffect) {
-                vibrationStrength = 8
-            }
-            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
-                vibrator?.vibrate(
-                    VibrationEffect.createOneShot(
-                        vibrationDuration, vibrationStrength
-                    )
-                )
-            } else {
-                @Suppress("DEPRECATION")
-                vibrator?.vibrate(vibrationDuration)
-            }
-        }
-    }
-
-    /**
-     * Makes a key press sound if the user has this feature enabled in the preferences.
-     */
-    fun keyPressSound(keyData: KeyData? = null) {
-        if (prefs.keyboard.soundEnabled) {
-            val soundVolume = prefs.keyboard.soundVolume
-            val effect = when (keyData) {
-                is TextKeyData -> when (keyData.code) {
-                    KeyCode.SPACE -> AudioManager.FX_KEYPRESS_SPACEBAR
-                    KeyCode.DELETE -> AudioManager.FX_KEYPRESS_DELETE
-                    KeyCode.ENTER -> AudioManager.FX_KEYPRESS_RETURN
-                    else -> AudioManager.FX_KEYPRESS_STANDARD
-                }
-                else -> AudioManager.FX_KEYPRESS_STANDARD
-            }
-            if (soundVolume == -1 && prefs.keyboard.soundEnabledSystem) {
-                audioManager!!.playSoundEffect(effect)
-            } else if (soundVolume > 0) {
-                audioManager!!.playSoundEffect(effect, soundVolume / 100f)
-            }
-        }
-    }
-
     /**
      * Executes a given [SwipeAction]. Ignores any [SwipeAction] but the ones relevant for this
      * class.
@@ -884,35 +801,35 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
     fun switchToPrevSubtype() {
         flogInfo(LogTopic.IMS_EVENTS)
         activeSubtype = subtypeManager.switchToPrevSubtype() ?: Subtype.DEFAULT
-        onSubtypeChanged(activeSubtype)
+        onSubtypeChanged(activeSubtype, true)
     }
 
     fun switchToNextSubtype() {
         flogInfo(LogTopic.IMS_EVENTS)
         activeSubtype = subtypeManager.switchToNextSubtype() ?: Subtype.DEFAULT
-        onSubtypeChanged(activeSubtype)
+        onSubtypeChanged(activeSubtype, true)
     }
 
-    private fun onSubtypeChanged(newSubtype: Subtype) {
+    private fun onSubtypeChanged(newSubtype: Subtype, doRefreshLayouts: Boolean) {
         flogInfo(LogTopic.SUBTYPE_MANAGER) { "New subtype: $newSubtype" }
-        textInputManager.onSubtypeChanged(newSubtype)
-        mediaInputManager.onSubtypeChanged(newSubtype)
-        clipInputManager.onSubtypeChanged(newSubtype)
+        textInputManager.onSubtypeChanged(newSubtype, doRefreshLayouts)
+        mediaInputManager.onSubtypeChanged(newSubtype, doRefreshLayouts)
+        clipInputManager.onSubtypeChanged(newSubtype, doRefreshLayouts)
     }
 
     fun setActiveInput(type: Int, forceSwitchToCharacters: Boolean = false) {
         when (type) {
             R.id.text_input -> {
-                inputView?.mainViewFlipper?.displayedChild = 0
+                uiBinding?.mainViewFlipper?.displayedChild = 0
                 if (forceSwitchToCharacters) {
                     textInputManager.inputEventDispatcher.send(InputKeyEvent.downUp(TextKeyData.VIEW_CHARACTERS))
                 }
             }
             R.id.media_input -> {
-                inputView?.mainViewFlipper?.displayedChild = 1
+                uiBinding?.mainViewFlipper?.displayedChild = 1
             }
             R.id.clip_input -> {
-                inputView?.mainViewFlipper?.displayedChild = 2
+                uiBinding?.mainViewFlipper?.displayedChild = 2
             }
         }
         textInputManager.isGlidePostEffect = false
@@ -928,27 +845,27 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
 
     fun updateOneHandedPanelVisibility() {
         if (resources.configuration.orientation != Configuration.ORIENTATION_PORTRAIT) {
-            inputView?.oneHandedCtrlPanelStart?.visibility = View.GONE
-            inputView?.oneHandedCtrlPanelEnd?.visibility = View.GONE
+            uiBinding?.oneHandedCtrlPanelStart?.visibility = View.GONE
+            uiBinding?.oneHandedCtrlPanelEnd?.visibility = View.GONE
         } else {
             when (prefs.keyboard.oneHandedMode) {
                 OneHandedMode.OFF -> {
-                    inputView?.oneHandedCtrlPanelStart?.visibility = View.GONE
-                    inputView?.oneHandedCtrlPanelEnd?.visibility = View.GONE
+                    uiBinding?.oneHandedCtrlPanelStart?.visibility = View.GONE
+                    uiBinding?.oneHandedCtrlPanelEnd?.visibility = View.GONE
                 }
                 OneHandedMode.START -> {
-                    inputView?.oneHandedCtrlPanelStart?.visibility = View.GONE
-                    inputView?.oneHandedCtrlPanelEnd?.visibility = View.VISIBLE
+                    uiBinding?.oneHandedCtrlPanelStart?.visibility = View.GONE
+                    uiBinding?.oneHandedCtrlPanelEnd?.visibility = View.VISIBLE
                 }
                 OneHandedMode.END -> {
-                    inputView?.oneHandedCtrlPanelStart?.visibility = View.VISIBLE
-                    inputView?.oneHandedCtrlPanelEnd?.visibility = View.GONE
+                    uiBinding?.oneHandedCtrlPanelStart?.visibility = View.VISIBLE
+                    uiBinding?.oneHandedCtrlPanelEnd?.visibility = View.GONE
                 }
             }
         }
         // Delay execution so this function can return, then refresh the whole layout
         uiScope.launch {
-            refreshLayoutOf(inputView)
+            refreshLayoutOf(uiBinding?.inputView)
         }
     }
 
@@ -981,8 +898,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
 
     interface EventListener {
         fun onCreate() {}
-        fun onCreateInputView() {}
-        fun onRegisterInputView(inputView: InputView) {}
+        fun onInitializeInputUi(uiBinding: FlorisboardBinding) {}
         fun onDestroy() {}
 
         fun onStartInputView(instance: EditorInstance, restarting: Boolean) {}
@@ -995,7 +911,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
 
         fun onApplyThemeAttributes() {}
         fun onPrimaryClipChanged() {}
-        fun onSubtypeChanged(newSubtype: Subtype) {}
+        fun onSubtypeChanged(newSubtype: Subtype, doRefreshLayouts: Boolean) {}
     }
 
     private enum class ResponseState {
@@ -1067,7 +983,7 @@ open class FlorisBoard : InputMethodService(), LifecycleOwner, FlorisClipboardMa
 
             val tmpSubtypeList = mutableListOf<Pair<String, String>>()
             for (defaultSubtype in defaultSubtypes) {
-                tmpSubtypeList.add(Pair(defaultSubtype.locale.toString(), defaultSubtype.locale.displayName))
+                tmpSubtypeList.add(Pair(defaultSubtype.locale.localeTag(), defaultSubtype.locale.displayName()))
             }
             // Sort language list alphabetically by the display name of a language
             tmpSubtypeList.sortBy { it.second }

app/src/main/java/dev/patrickgold/florisboard/ime/core/InputEventDispatcher.kt

@@ -20,12 +20,12 @@ import android.os.SystemClock
 import android.util.SparseArray
 import androidx.core.util.forEach
 import androidx.core.util.set
-import dev.patrickgold.florisboard.BuildConfig
+import dev.patrickgold.florisboard.debug.LogTopic
+import dev.patrickgold.florisboard.debug.flogDebug
+import dev.patrickgold.florisboard.ime.keyboard.KeyData
 import dev.patrickgold.florisboard.ime.text.key.KeyCode
-import dev.patrickgold.florisboard.ime.text.keyboard.TextKeyData
 import kotlinx.coroutines.*
 import kotlinx.coroutines.channels.Channel
-import timber.log.Timber
 
 /**
  * The main logic point of processing input events and delegating them to the registered event receivers. Currently,
@@ -57,7 +57,7 @@ class InputEventDispatcher private constructor(
         /**
          * The default input event channel capacity to be used in [new].
          */
-        private const val DEFAULT_CHANNEL_CAPACITY: Int = 32
+        private const val DEFAULT_CHANNEL_CAPACITY: Int = 64
 
         /**
          * Creates a new [InputEventDispatcher] instance from given arguments and returns it.
@@ -96,9 +96,7 @@ class InputEventDispatcher private constructor(
             for (ev in channel) {
                 if (!isActive) break
                 val startTime = System.nanoTime()
-                if (BuildConfig.DEBUG) {
-                    Timber.d(ev.toString())
-                }
+                flogDebug(LogTopic.KEY_EVENTS) { ev.toString() }
                 when (ev.action) {
                     InputKeyEvent.Action.DOWN -> {
                         if (pressedKeys.indexOfKey(ev.data.code) >= 0) continue
@@ -155,9 +153,7 @@ class InputEventDispatcher private constructor(
                         }
                     }
                 }
-                if (BuildConfig.DEBUG) {
-                    Timber.d("Time elapsed: ${(System.nanoTime() - startTime) / 1_000_000}")
-                }
+                flogDebug(LogTopic.KEY_EVENTS) { "Time elapsed: ${(System.nanoTime() - startTime) / 1_000_000}" }
             }
             pressedKeys.forEach { _, value -> value.repeatKeyPressJob?.cancel() }
             pressedKeys.clear()
@@ -210,7 +206,7 @@ class InputEventDispatcher private constructor(
 data class InputKeyEvent(
     val eventTime: Long,
     val action: Action,
-    val data: TextKeyData,
+    val data: KeyData,
     val count: Int
 ) {
     companion object {
@@ -221,7 +217,7 @@ data class InputKeyEvent(
          *
          * @return The created input key event.
          */
-        fun down(keyData: TextKeyData): InputKeyEvent {
+        fun down(keyData: KeyData): InputKeyEvent {
             return InputKeyEvent(
                 eventTime = SystemClock.uptimeMillis(),
                 action = Action.DOWN,
@@ -238,7 +234,7 @@ data class InputKeyEvent(
          *
          * @return The created input key event.
          */
-        fun downUp(keyData: TextKeyData, count: Int = 1): InputKeyEvent {
+        fun downUp(keyData: KeyData, count: Int = 1): InputKeyEvent {
             return InputKeyEvent(
                 eventTime = SystemClock.uptimeMillis(),
                 action = Action.DOWN_UP,
@@ -254,7 +250,7 @@ data class InputKeyEvent(
          *
          * @return The created input key event.
          */
-        fun up(keyData: TextKeyData): InputKeyEvent {
+        fun up(keyData: KeyData): InputKeyEvent {
             return InputKeyEvent(
                 eventTime = SystemClock.uptimeMillis(),
                 action = Action.UP,
@@ -271,7 +267,7 @@ data class InputKeyEvent(
          *
          * @return The created input key event.
          */
-        fun repeat(keyData: TextKeyData, count: Int = 1): InputKeyEvent {
+        fun repeat(keyData: KeyData, count: Int = 1): InputKeyEvent {
             return InputKeyEvent(
                 eventTime = SystemClock.uptimeMillis(),
                 action = Action.REPEAT,
@@ -287,7 +283,7 @@ data class InputKeyEvent(
          *
          * @return The created input key event.
          */
-        fun cancel(keyData: TextKeyData): InputKeyEvent {
+        fun cancel(keyData: KeyData): InputKeyEvent {
             return InputKeyEvent(
                 eventTime = SystemClock.uptimeMillis(),
                 action = Action.CANCEL,

app/src/main/java/dev/patrickgold/florisboard/ime/core/InputView.kt

@@ -26,19 +26,16 @@ import android.os.Build
 import android.text.TextPaint
 import android.util.AttributeSet
 import android.util.DisplayMetrics
-import android.view.ViewGroup
 import android.widget.LinearLayout
-import android.widget.ViewFlipper
 import dev.patrickgold.florisboard.R
 import dev.patrickgold.florisboard.ime.onehanded.OneHandedMode
 import dev.patrickgold.florisboard.ime.text.key.KeyVariation
 import dev.patrickgold.florisboard.ime.text.keyboard.KeyboardMode
 import dev.patrickgold.florisboard.common.ViewUtils
-import timber.log.Timber
 import kotlin.math.roundToInt
 
 /**
- * Root view of the keyboard. Notifies [FlorisBoard] when it has been attached to a window.
+ * Root view of the keyboard.
  */
 class InputView : LinearLayout {
     private val florisboard get() = FlorisBoard.getInstance()
@@ -66,13 +63,6 @@ class InputView : LinearLayout {
     var desiredInlineSuggestionsMaxHeight: Int = 0
         private set
 
-    var mainViewFlipper: ViewFlipper? = null
-        private set
-    var oneHandedCtrlPanelStart: ViewGroup? = null
-        private set
-    var oneHandedCtrlPanelEnd: ViewGroup? = null
-        private set
-
     private val overlayTextPaint: TextPaint = TextPaint().apply {
         color = Color.GREEN
         textAlign = Paint.Align.RIGHT
@@ -88,18 +78,6 @@ class InputView : LinearLayout {
         defStyleAttr
     )
 
-    override fun onAttachedToWindow() {
-        Timber.i("onAttachedToWindow()")
-
-        super.onAttachedToWindow()
-
-        mainViewFlipper = findViewById(R.id.main_view_flipper)
-        oneHandedCtrlPanelStart = findViewById(R.id.one_handed_ctrl_panel_start)
-        oneHandedCtrlPanelEnd = findViewById(R.id.one_handed_ctrl_panel_end)
-
-        florisboard.registerInputView(this)
-    }
-
     override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
         heightFactor = when (resources.configuration.orientation) {
             Configuration.ORIENTATION_LANDSCAPE -> 1.0f

app/src/main/java/dev/patrickgold/florisboard/ime/core/InputWindowView.kt

@@ -21,10 +21,17 @@ import android.util.AttributeSet
 import android.widget.FrameLayout
 
 /**
- * Root view of the keyboard.
+ * Root window view of the keyboard.
  */
 class InputWindowView : FrameLayout {
+    private val florisboard get() = FlorisBoard.getInstanceOrNull()
+
     constructor(context: Context) : this(context, null)
     constructor(context: Context, attrs: AttributeSet?) : this(context, attrs, 0)
     constructor(context: Context, attrs: AttributeSet?, defStyleAttr: Int) : super(context, attrs, defStyleAttr)
+
+    override fun onAttachedToWindow() {
+        super.onAttachedToWindow()
+        florisboard?.initWindow()
+    }
 }

app/src/main/java/dev/patrickgold/florisboard/ime/core/Preferences.kt

@@ -18,7 +18,8 @@ package dev.patrickgold.florisboard.ime.core
 
 import android.content.Context
 import android.content.SharedPreferences
-import android.provider.Settings
+import android.os.Build
+import androidx.core.os.UserManagerCompat
 import androidx.preference.PreferenceManager
 import dev.patrickgold.florisboard.R
 import dev.patrickgold.florisboard.ime.landscapeinput.LandscapeInputUiMode
@@ -40,13 +41,13 @@ import java.lang.ref.WeakReference
  */
 class Preferences(
     context: Context,
-    val shared: SharedPreferences = PreferenceManager.getDefaultSharedPreferences(context)
 ) {
-    private val applicationContext: WeakReference<Context> = WeakReference(context.applicationContext)
+    var shared: SharedPreferences = if (!UserManagerCompat.isUserUnlocked(context) && Build.VERSION.SDK_INT >= Build.VERSION_CODES.N)
+        context.createDeviceProtectedStorageContext().getSharedPreferences("shared_psfs", Context.MODE_PRIVATE)
+    else
+        PreferenceManager.getDefaultSharedPreferences(context)
 
-    private val cacheBoolean: HashMap<String, Boolean> = hashMapOf()
-    private val cacheInt: HashMap<String, Int> = hashMapOf()
-    private val cacheString: HashMap<String, String> = hashMapOf()
+    private val applicationContext: WeakReference<Context> = WeakReference(context.applicationContext)
 
     val advanced = Advanced(this)
     val clipboard = Clipboard(this)
@@ -55,54 +56,31 @@ class Preferences(
     val dictionary = Dictionary(this)
     val gestures = Gestures(this)
     val glide = Glide(this)
+    val inputFeedback = InputFeedback(this)
     val internal = Internal(this)
     val keyboard = Keyboard(this)
     val localization = Localization(this)
     val smartbar = Smartbar(this)
+    val spelling = Spelling(this)
     val suggestion = Suggestion(this)
     val theme = Theme(this)
 
+
     /**
-     * Checks the cache if an entry for [key] exists, else calls [getPrefInternal] to retrieve the
-     * value. The type is automatically derived from the given [default] value.
+     * Gets the value for given [key]. The type is automatically derived from the given [default] value.
+     *
      * @return The value for [key] or [default].
      */
     private inline fun <reified T> getPref(key: String, default: T): T {
         return when {
             false is T -> {
-                (cacheBoolean[key] ?: getPrefInternal(key, default)) as T
+                shared.getBoolean(key, default as Boolean) as T
             }
             0 is T -> {
-                (cacheInt[key] ?: getPrefInternal(key, default)) as T
+                shared.getInt(key, default as Int) as T
             }
             "" is T -> {
-                (cacheString[key] ?: getPrefInternal(key, default)) as T
-            }
-            else -> null as T
-        }
-    }
-
-    /**
-     * Fetches the value for [key] from the shared preferences, puts the value into the
-     * corresponding cache and returns it.
-     * @return The value for [key] or [default].
-     */
-    private inline fun <reified T> getPrefInternal(key: String, default: T): T {
-        return when {
-            false is T -> {
-                val value = shared.getBoolean(key, default as Boolean)
-                cacheBoolean[key] = value
-                value as T
-            }
-            0 is T -> {
-                val value = shared.getInt(key, default as Int)
-                cacheInt[key] = value
-                value as T
-            }
-            "" is T -> {
-                val value = (shared.getString(key, default as String) ?: (default as String))
-                cacheString[key] = value
-                value as T
+                (shared.getString(key, default as String) ?: (default as String)) as T
             }
             else -> null as T
         }
@@ -116,15 +94,12 @@ class Preferences(
         when {
             false is T -> {
                 shared.edit().putBoolean(key, value as Boolean).apply()
-                cacheBoolean[key] = value as Boolean
             }
             0 is T -> {
                 shared.edit().putInt(key, value as Int).apply()
-                cacheInt[key] = value as Int
             }
             "" is T -> {
                 shared.edit().putString(key, value as String).apply()
-                cacheString[key] = value as String
             }
         }
     }
@@ -156,42 +131,27 @@ class Preferences(
      * they have not been initialized yet.
      */
     fun initDefaultPreferences() {
-        applicationContext.get()?.let { context ->
-            PreferenceManager.setDefaultValues(context, R.xml.prefs_advanced, true)
-            PreferenceManager.setDefaultValues(context, R.xml.prefs_gestures, true)
-            PreferenceManager.setDefaultValues(context, R.xml.prefs_keyboard, true)
-            PreferenceManager.setDefaultValues(context, R.xml.prefs_theme, true)
-            PreferenceManager.setDefaultValues(context, R.xml.prefs_typing, true)
-        }
+        try {
+            applicationContext.get()?.let { context ->
+                PreferenceManager.setDefaultValues(context, R.xml.prefs_advanced, true)
+                PreferenceManager.setDefaultValues(context, R.xml.prefs_gestures, true)
+                PreferenceManager.setDefaultValues(context, R.xml.prefs_keyboard, true)
+                PreferenceManager.setDefaultValues(context, R.xml.prefs_theme, true)
+                PreferenceManager.setDefaultValues(context, R.xml.prefs_typing, true)
+            }
 
-        //theme.dayThemeRef = "assets:ime/theme/floris_day.json"
-        //theme.nightThemeRef = "assets:ime/theme/floris_night.json"
-        //setPref(Localization.SUBTYPES, "-234/de-AT/euro/c=qwertz")
-        val subtypes = getPref(Localization.SUBTYPES, "")
-        if (subtypes.matches(OLD_SUBTYPES_REGEX)) {
-            setPref(Localization.SUBTYPES, "")
+            //theme.dayThemeRef = "assets:ime/theme/floris_day.json"
+            //theme.nightThemeRef = "assets:ime/theme/floris_night.json"
+            //setPref(Localization.SUBTYPES, "-234/de-AT/euro/c=qwertz")
+            val subtypes = getPref(Localization.SUBTYPES, "")
+            if (subtypes.matches(OLD_SUBTYPES_REGEX)) {
+                setPref(Localization.SUBTYPES, "")
+            }
+        } catch (e: Exception) {
+            e.fillInStackTrace()
         }
     }
 
-    /**
-     * Syncs the system preference values and clears the cache.
-     */
-    fun sync() {
-        applicationContext.get()?.let { context ->
-            val contentResolver = context.contentResolver
-            keyboard.soundEnabledSystem = Settings.System.getInt(
-                contentResolver, Settings.System.SOUND_EFFECTS_ENABLED, 0
-            ) != 0
-            keyboard.vibrationEnabledSystem = Settings.System.getInt(
-                contentResolver, Settings.System.HAPTIC_FEEDBACK_ENABLED, 0
-            ) != 0
-        }
-
-        cacheBoolean.clear()
-        cacheInt.clear()
-        cacheString.clear()
-    }
-
     /**
      * Wrapper class for advanced preferences.
      */
@@ -220,6 +180,7 @@ class Preferences(
         companion object {
             const val AUTO_CAPITALIZATION =         "correction__auto_capitalization"
             const val DOUBLE_SPACE_PERIOD =         "correction__double_space_period"
+            const val MANAGE_SPELL_CHECKER =        "correction__manage_spell_checker"
             const val REMEMBER_CAPS_LOCK_STATE =    "correction__remember_caps_lock_state"
         }
 
@@ -334,7 +295,6 @@ class Preferences(
             const val TRAIL_DURATION =              "glide__trail_fade_duration"
             const val SHOW_PREVIEW =                "glide__show_preview"
             const val PREVIEW_REFRESH_DELAY =       "glide__preview_refresh_delay"
-            const val MAX_TRAIL_LENGTH  =           "glide__trail_max_length"
         }
 
         var enabled: Boolean
@@ -352,9 +312,90 @@ class Preferences(
         var previewRefreshDelay: Int
             get() = prefs.getPref(PREVIEW_REFRESH_DELAY, 150)
             set(v) = prefs.setPref(PREVIEW_REFRESH_DELAY, v)
-        var trailMaxLength: Int
-            get() = prefs.getPref(MAX_TRAIL_LENGTH, 150)
-            set(v) = prefs.setPref(MAX_TRAIL_LENGTH, v)
+    }
+
+    /**
+     * Wrapper class for internal preferences. A preference qualifies as an internal pref if the
+     * user has no ability to control this preference's value directly (via a UI pref view).
+     */
+    class InputFeedback(private val prefs: Preferences) {
+        companion object {
+            const val AUDIO_ENABLED =                       "input_feedback__audio_enabled"
+            const val AUDIO_IGNORE_SYSTEM_SETTINGS =        "input_feedback__audio_ignore_system_settings"
+            const val AUDIO_VOLUME =                        "input_feedback__audio_volume"
+            const val AUDIO_FEAT_KEY_PRESS =                "input_feedback__audio_feat_key_press"
+            const val AUDIO_FEAT_KEY_LONG_PRESS =           "input_feedback__audio_feat_key_long_press"
+            const val AUDIO_FEAT_KEY_REPEATED_ACTION =      "input_feedback__audio_feat_key_repeated_action"
+            const val AUDIO_FEAT_GESTURE_SWIPE =            "input_feedback__audio_feat_gesture_swipe"
+            const val AUDIO_FEAT_GESTURE_MOVING_SWIPE =     "input_feedback__audio_feat_gesture_moving_swipe"
+
+            const val HAPTIC_ENABLED =                      "input_feedback__haptic_enabled"
+            const val HAPTIC_IGNORE_SYSTEM_SETTINGS =       "input_feedback__haptic_ignore_system_settings"
+            const val HAPTIC_USE_VIBRATOR =                 "input_feedback__haptic_use_vibrator"
+            const val HAPTIC_VIBRATION_DURATION =           "input_feedback__haptic_vibration_duration"
+            const val HAPTIC_VIBRATION_STRENGTH =           "input_feedback__haptic_vibration_strength"
+            const val HAPTIC_FEAT_KEY_PRESS =               "input_feedback__haptic_feat_key_press"
+            const val HAPTIC_FEAT_KEY_LONG_PRESS =          "input_feedback__haptic_feat_key_long_press"
+            const val HAPTIC_FEAT_KEY_REPEATED_ACTION =     "input_feedback__haptic_feat_key_repeated_action"
+            const val HAPTIC_FEAT_GESTURE_SWIPE =           "input_feedback__haptic_feat_gesture_swipe"
+            const val HAPTIC_FEAT_GESTURE_MOVING_SWIPE =    "input_feedback__haptic_feat_gesture_moving_swipe"
+        }
+
+        var audioEnabled: Boolean
+            get() =  prefs.getPref(AUDIO_ENABLED, true)
+            set(v) = prefs.setPref(AUDIO_ENABLED, v)
+        var audioIgnoreSystemSettings: Boolean
+            get() =  prefs.getPref(AUDIO_IGNORE_SYSTEM_SETTINGS, false)
+            set(v) = prefs.setPref(AUDIO_IGNORE_SYSTEM_SETTINGS, v)
+        var audioVolume: Int
+            get() =  prefs.getPref(AUDIO_VOLUME, 50)
+            set(v) = prefs.setPref(AUDIO_VOLUME, v)
+        var audioFeatKeyPress: Boolean
+            get() =  prefs.getPref(AUDIO_FEAT_KEY_PRESS, true)
+            set(v) = prefs.setPref(AUDIO_FEAT_KEY_PRESS, v)
+        var audioFeatKeyLongPress: Boolean
+            get() =  prefs.getPref(AUDIO_FEAT_KEY_LONG_PRESS, false)
+            set(v) = prefs.setPref(AUDIO_FEAT_KEY_LONG_PRESS, v)
+        var audioFeatKeyRepeatedAction: Boolean
+            get() =  prefs.getPref(AUDIO_FEAT_KEY_REPEATED_ACTION, false)
+            set(v) = prefs.setPref(AUDIO_FEAT_KEY_REPEATED_ACTION, v)
+        var audioFeatGestureSwipe: Boolean
+            get() =  prefs.getPref(AUDIO_FEAT_GESTURE_SWIPE, false)
+            set(v) = prefs.setPref(AUDIO_FEAT_GESTURE_SWIPE, v)
+        var audioFeatGestureMovingSwipe: Boolean
+            get() =  prefs.getPref(AUDIO_FEAT_GESTURE_MOVING_SWIPE, false)
+            set(v) = prefs.setPref(AUDIO_FEAT_GESTURE_MOVING_SWIPE, v)
+
+        var hapticEnabled: Boolean
+            get() =  prefs.getPref(HAPTIC_ENABLED, true)
+            set(v) = prefs.setPref(HAPTIC_ENABLED, v)
+        var hapticIgnoreSystemSettings: Boolean
+            get() =  prefs.getPref(HAPTIC_IGNORE_SYSTEM_SETTINGS, false)
+            set(v) = prefs.setPref(HAPTIC_IGNORE_SYSTEM_SETTINGS, v)
+        var hapticUseVibrator: Boolean
+            get() =  prefs.getPref(HAPTIC_USE_VIBRATOR, true)
+            set(v) = prefs.setPref(HAPTIC_USE_VIBRATOR, v)
+        var hapticVibrationDuration: Int
+            get() =  prefs.getPref(HAPTIC_VIBRATION_DURATION, 50)
+            set(v) = prefs.setPref(HAPTIC_VIBRATION_DURATION, v)
+        var hapticVibrationStrength: Int
+            get() =  prefs.getPref(HAPTIC_VIBRATION_STRENGTH, 50)
+            set(v) = prefs.setPref(HAPTIC_VIBRATION_STRENGTH, v)
+        var hapticFeatKeyPress: Boolean
+            get() =  prefs.getPref(HAPTIC_FEAT_KEY_PRESS, true)
+            set(v) = prefs.setPref(HAPTIC_FEAT_KEY_PRESS, v)
+        var hapticFeatKeyLongPress: Boolean
+            get() =  prefs.getPref(HAPTIC_FEAT_KEY_LONG_PRESS, false)
+            set(v) = prefs.setPref(HAPTIC_FEAT_KEY_LONG_PRESS, v)
+        var hapticFeatKeyRepeatedAction: Boolean
+            get() =  prefs.getPref(HAPTIC_FEAT_KEY_REPEATED_ACTION, true)
+            set(v) = prefs.setPref(HAPTIC_FEAT_KEY_REPEATED_ACTION, v)
+        var hapticFeatGestureSwipe: Boolean
+            get() =  prefs.getPref(HAPTIC_FEAT_GESTURE_SWIPE, false)
+            set(v) = prefs.setPref(HAPTIC_FEAT_GESTURE_SWIPE, v)
+        var hapticFeatGestureMovingSwipe: Boolean
+            get() =  prefs.getPref(HAPTIC_FEAT_GESTURE_MOVING_SWIPE, true)
+            set(v) = prefs.setPref(HAPTIC_FEAT_GESTURE_MOVING_SWIPE, v)
     }
 
     /**
@@ -405,14 +446,9 @@ class Preferences(
             const val ONE_HANDED_MODE =                     "keyboard__one_handed_mode"
             const val ONE_HANDED_MODE_SCALE_FACTOR =        "keyboard__one_handed_mode_scale_factor"
             const val POPUP_ENABLED =                       "keyboard__popup_enabled"
-            const val SOUND_ENABLED =                       "keyboard__sound_enabled"
-            const val SOUND_VOLUME =                        "keyboard__sound_volume"
             const val SPACE_BAR_SWITCHES_TO_CHARACTERS =    "keyboard__space_bar_switches_to_characters"
             const val UTILITY_KEY_ACTION =                  "keyboard__utility_key_action"
             const val UTILITY_KEY_ENABLED =                 "keyboard__utility_key_enabled"
-            const val VIBRATION_ENABLED =                   "keyboard__vibration_enabled"
-            const val VIBRATION_DURATION =                  "keyboard__vibration_duration"
-            const val VIBRATION_STRENGTH =                  "keyboard__vibration_strength"
         }
 
         var bottomOffsetPortrait: Int = 0
@@ -466,13 +502,6 @@ class Preferences(
         var popupEnabled: Boolean = false
             get() = prefs.getPref(POPUP_ENABLED, true)
             private set
-        var soundEnabled: Boolean = false
-            get() = prefs.getPref(SOUND_ENABLED, true)
-            private set
-        var soundEnabledSystem: Boolean = false
-        var soundVolume: Int = 0
-            get() = prefs.getPref(SOUND_VOLUME, -1)
-            private set
         var spaceBarSwitchesToCharacters: Boolean
             get() =  prefs.getPref(SPACE_BAR_SWITCHES_TO_CHARACTERS, true)
             set(v) = prefs.setPref(SPACE_BAR_SWITCHES_TO_CHARACTERS, v)
@@ -482,16 +511,6 @@ class Preferences(
         var utilityKeyEnabled: Boolean
             get() =  prefs.getPref(UTILITY_KEY_ENABLED, true)
             set(v) = prefs.setPref(UTILITY_KEY_ENABLED, v)
-        var vibrationEnabled: Boolean = false
-            get() = prefs.getPref(VIBRATION_ENABLED, true)
-            private set
-        var vibrationEnabledSystem: Boolean = false
-        var vibrationDuration: Int = 0
-            get() = prefs.getPref(VIBRATION_DURATION, -1)
-            private set
-        var vibrationStrength: Int = 0
-            get() = prefs.getPref(VIBRATION_STRENGTH, -1)
-            private set
 
         fun keyHintConfiguration(): KeyHintConfiguration {
             return KeyHintConfiguration(hintedSymbolsMode, hintedNumberRowMode, mergeHintPopupsEnabled)
@@ -528,6 +547,25 @@ class Preferences(
             set(v) = prefs.setPref(ENABLED, v)
     }
 
+    /**
+     * Wrapper class for Spelling preferences.
+     */
+    class Spelling(private val prefs: Preferences) {
+        companion object {
+            const val ACTIVE_SPELLCHECKER =         "spelling__active_spellchecker"
+            const val MANAGE_DICTIONARIES =         "spelling__manage_dictionaries"
+            const val USE_CONTACTS =                "spelling__use_contacts"
+            const val USE_UDM_ENTRIES =             "spelling__use_udm_entries"
+        }
+
+        var useContacts: Boolean
+            get() =  prefs.getPref(USE_CONTACTS, true)
+            set(v) = prefs.setPref(USE_CONTACTS, v)
+        var useUdmEntries: Boolean
+            get() =  prefs.getPref(USE_UDM_ENTRIES, true)
+            set(v) = prefs.setPref(USE_UDM_ENTRIES, v)
+    }
+
     /**
      * Wrapper class for suggestion preferences.
      */

app/src/main/java/dev/patrickgold/florisboard/ime/core/Subtype.kt

@@ -16,16 +16,10 @@
 
 package dev.patrickgold.florisboard.ime.core
 
+import dev.patrickgold.florisboard.common.FlorisLocale
 import dev.patrickgold.florisboard.ime.text.composing.Appender
-import dev.patrickgold.florisboard.ime.text.composing.Composer
 import dev.patrickgold.florisboard.ime.text.layout.LayoutType
-import dev.patrickgold.florisboard.util.LocaleUtils
 import kotlinx.serialization.*
-import kotlinx.serialization.descriptors.PrimitiveKind
-import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
-import kotlinx.serialization.descriptors.SerialDescriptor
-import kotlinx.serialization.encoding.Decoder
-import kotlinx.serialization.encoding.Encoder
 import java.util.*
 
 /**
@@ -40,7 +34,7 @@ import java.util.*
  */
 data class Subtype(
     val id: Int,
-    val locale: Locale,
+    val locale: FlorisLocale,
     val composerName: String,
     val currencySetName: String,
     val layoutMap: SubtypeLayoutMap,
@@ -53,7 +47,7 @@ data class Subtype(
          */
         val DEFAULT = Subtype(
             id = -1,
-            locale = Locale.ENGLISH,
+            locale = FlorisLocale.ENGLISH,
             composerName = Appender.name,
             currencySetName = "\$default",
             layoutMap = SubtypeLayoutMap(characters = "qwerty")
@@ -74,7 +68,7 @@ data class Subtype(
             val data = str.split("/")
             when (data.size) {
                 4 -> {
-                    val locale = LocaleUtils.stringToLocale(data[1])
+                    val locale = FlorisLocale.fromTag(data[1])
                     return Subtype(
                         data[0].toInt(),
                         locale,
@@ -84,7 +78,7 @@ data class Subtype(
                     )
                 }
                 5 -> {
-                    val locale = LocaleUtils.stringToLocale(data[1])
+                    val locale = FlorisLocale.fromTag(data[1])
                     return Subtype(
                         data[0].toInt(),
                         locale,
@@ -114,7 +108,7 @@ data class Subtype(
      *  <id>/<language_tag>/<composer_name>/<currency_set_name>/<layout_map>
      */
     override fun toString(): String {
-        val languageTag = locale.toLanguageTag()
+        val languageTag = locale.languageTag()
         return "$id/$languageTag/$composerName/$currencySetName/$layoutMap"
     }
 
@@ -123,7 +117,7 @@ data class Subtype(
      *  <id>/<language_tag>/<currency_set_name>
      */
     fun toShortString(): String {
-        val languageTag = locale.toLanguageTag()
+        val languageTag = locale.languageTag()
         return "$id/$languageTag/$currencySetName"
     }
 
@@ -326,24 +320,12 @@ data class SubtypeLayoutMap(
 @Serializable
 data class DefaultSubtype(
     var id: Int,
-    @Serializable(with = LocaleSerializer::class)
+    @Serializable(with = FlorisLocale.Serializer::class)
     @SerialName("languageTag")
-    var locale: Locale,
+    var locale: FlorisLocale,
     @SerialName("composer")
     var composerName: String,
     @SerialName("currencySet")
     var currencySetName: String,
     var preferred: SubtypeLayoutMap
 )
-
-class LocaleSerializer : KSerializer<Locale> {
-    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("Locale", PrimitiveKind.STRING)
-
-    override fun serialize(encoder: Encoder, value: Locale) {
-        encoder.encodeString(value.toString())
-    }
-
-    override fun deserialize(decoder: Decoder): Locale {
-        return LocaleUtils.stringToLocale(decoder.decodeString())
-    }
-}

app/src/main/java/dev/patrickgold/florisboard/ime/core/SubtypeManager.kt

@@ -17,14 +17,13 @@
 package dev.patrickgold.florisboard.ime.core
 
 import android.content.Context
+import dev.patrickgold.florisboard.common.FlorisLocale
 import dev.patrickgold.florisboard.debug.*
-import dev.patrickgold.florisboard.ime.extension.AssetManager
-import dev.patrickgold.florisboard.ime.extension.AssetRef
-import dev.patrickgold.florisboard.ime.extension.AssetSource
+import dev.patrickgold.florisboard.res.AssetManager
 import dev.patrickgold.florisboard.ime.text.key.CurrencySet
+import dev.patrickgold.florisboard.res.FlorisRef
 import kotlinx.coroutines.CoroutineScope
 import kotlinx.coroutines.MainScope
-import java.util.*
 import kotlin.collections.ArrayList
 
 /**
@@ -85,7 +84,7 @@ class SubtypeManager(
      * @return The [FlorisBoard.ImeConfig] or a default config.
      */
     private fun loadImeConfig(path: String): FlorisBoard.ImeConfig {
-        return assetManager.loadJsonAsset<FlorisBoard.ImeConfig>(AssetRef(AssetSource.Assets, path)).getOrElse {
+        return assetManager.loadJsonAsset<FlorisBoard.ImeConfig>(FlorisRef.assets(path)).getOrElse {
             flogError(LogTopic.SUBTYPE_MANAGER) { "Failed to retrieve IME config: $it" }
             FlorisBoard.ImeConfig(packageName)
         }
@@ -118,7 +117,7 @@ class SubtypeManager(
      * @return True if the subtype was added, false otherwise. A return value of false indicates
      *  that the subtype already exists.
      */
-    fun addSubtype(locale: Locale, composerName: String, currencySetName: String, layoutMap: SubtypeLayoutMap): Boolean {
+    fun addSubtype(locale: FlorisLocale, composerName: String, currencySetName: String, layoutMap: SubtypeLayoutMap): Boolean {
         return addSubtype(
             Subtype(
                 (locale.hashCode() + 31 * layoutMap.hashCode() + 31 * currencySetName.hashCode()),
@@ -186,7 +185,7 @@ class SubtypeManager(
      * @return The default system locale or null, if no matching default system subtype could be
      *  found.
      */
-    fun getDefaultSubtypeForLocale(locale: Locale): DefaultSubtype? {
+    fun getDefaultSubtypeForLocale(locale: FlorisLocale): DefaultSubtype? {
         for (defaultSubtype in imeConfig.defaultSubtypes) {
             if (defaultSubtype.locale == locale) {
                 return defaultSubtype

app/src/main/java/dev/patrickgold/florisboard/ime/core/TextProcessor.kt

@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.ime.core
+
+import dev.patrickgold.florisboard.common.FlorisLocale
+
+@Suppress("RegExpRedundantEscape")
+object TextProcessor {
+    private val LATIN_BASIC_WORD_REGEX = """[_]*(([\p{L}\d\']+[_-]*[\p{L}\d\']+)|[\p{L}\d\']+)[_]*""".toRegex()
+
+    private fun wordRegexFor(locale: FlorisLocale): Regex {
+        return when (locale) {
+            else -> LATIN_BASIC_WORD_REGEX
+        }
+    }
+
+    fun detectWords(text: CharSequence, locale: FlorisLocale): Sequence<IntRange> {
+        val regex = wordRegexFor(locale)
+        return regex.findAll(text).map { it.range }
+    }
+
+    fun detectWords(text: CharSequence, start: Int, end: Int, locale: FlorisLocale): Sequence<IntRange> {
+        val regex = wordRegexFor(locale)
+        val tStart = start.coerceAtLeast(0)
+        val tEnd = end.coerceAtMost(text.length)
+        return regex.findAll(text.slice(tStart..tEnd)).map { it.range }
+    }
+
+    fun isWord(text: CharSequence, locale: FlorisLocale): Boolean {
+        val regex = wordRegexFor(locale)
+        return regex.matches(text)
+    }
+}

app/src/main/java/dev/patrickgold/florisboard/ime/dictionary/Dictionary.kt

@@ -16,7 +16,7 @@
 
 package dev.patrickgold.florisboard.ime.dictionary
 
-import dev.patrickgold.florisboard.ime.extension.Asset
+import dev.patrickgold.florisboard.res.Asset
 import dev.patrickgold.florisboard.ime.nlp.SuggestionList
 import dev.patrickgold.florisboard.ime.nlp.Word
 

app/src/main/java/dev/patrickgold/florisboard/ime/dictionary/DictionaryManager.kt

@@ -18,16 +18,16 @@ package dev.patrickgold.florisboard.ime.dictionary
 
 import android.content.Context
 import androidx.room.Room
+import dev.patrickgold.florisboard.common.FlorisLocale
+import dev.patrickgold.florisboard.debug.flogError
 import dev.patrickgold.florisboard.ime.core.Preferences
 import dev.patrickgold.florisboard.ime.core.Subtype
-import dev.patrickgold.florisboard.ime.extension.AssetRef
 import dev.patrickgold.florisboard.ime.nlp.SuggestionList
 import dev.patrickgold.florisboard.ime.nlp.Word
+import dev.patrickgold.florisboard.res.FlorisRef
 import kotlinx.coroutines.CoroutineDispatcher
 import kotlinx.coroutines.Dispatchers
-import timber.log.Timber
 import java.lang.ref.WeakReference
-import java.util.*
 
 /**
  * TODO: document
@@ -45,6 +45,8 @@ class DictionaryManager private constructor(
     private var systemUserDictionaryDatabase: SystemUserDictionaryDatabase? = null
 
     companion object {
+        val FLORIS_EN_REF = FlorisRef.assets("ime/dict/en.flict")
+
         private var defaultInstance: DictionaryManager? = null
 
         fun init(applicationContext: Context): DictionaryManager {
@@ -75,15 +77,39 @@ class DictionaryManager private constructor(
     ) {
         val suggestions = SuggestionList.new(maxSuggestionCount)
         queryUserDictionary(currentWord, subtype.locale, suggestions)
+        loadDictionary(FLORIS_EN_REF).onSuccess {
+            it.getTokenPredictions(preceidingWords, currentWord, maxSuggestionCount, allowPossiblyOffensive, suggestions)
+        }
         block(suggestions)
         suggestions.dispose()
     }
 
-    fun prepareDictionaries(subtype: Subtype) {
-        // TODO: Implement this
+    fun loadDictionary(ref: FlorisRef): Result<Dictionary> {
+        dictionaryCache[ref.toString()]?.let {
+            return Result.success(it)
+        }
+        if (ref.relativePath.endsWith(".flict")) {
+            // Assume this is a Flictionary
+            applicationContext.get()?.let {
+                Flictionary.load(it, ref).onSuccess { flict ->
+                    dictionaryCache[ref.toString()] = flict
+                    return Result.success(flict)
+                }.onFailure { err ->
+                    flogError { err.toString() }
+                    return Result.failure(err)
+                }
+            }
+        } else {
+            return Result.failure(Exception("Unable to determine supported type for given AssetRef!"))
+        }
+        return Result.failure(Exception("If this message is ever thrown, something is completely broken..."))
     }
 
-    fun queryUserDictionary(word: Word, locale: Locale, destSuggestionList: SuggestionList) {
+    fun prepareDictionaries(subtype: Subtype) {
+        loadDictionary(FLORIS_EN_REF)
+    }
+
+    fun queryUserDictionary(word: Word, locale: FlorisLocale, destSuggestionList: SuggestionList) {
         val florisDao = florisUserDictionaryDao()
         val systemDao = systemUserDictionaryDao()
         if (florisDao == null && systemDao == null) {
@@ -115,9 +141,27 @@ class DictionaryManager private constructor(
         }
     }
 
+    fun spell(word: Word, locale: FlorisLocale): Boolean {
+        val florisDao = florisUserDictionaryDao()
+        val systemDao = systemUserDictionaryDao()
+        if (florisDao == null && systemDao == null) {
+            return false
+        }
+        var ret = false
+        if (prefs.dictionary.enableFlorisUserDictionary) {
+            ret = ret || florisDao?.queryExactFuzzyLocale(word, locale)?.isNotEmpty() ?: false
+            ret = ret || florisDao?.queryShortcut(word, locale)?.isNotEmpty() ?: false
+        }
+        if (prefs.dictionary.enableSystemUserDictionary) {
+            ret = ret || systemDao?.queryExactFuzzyLocale(word, locale)?.isNotEmpty() ?: false
+            ret = ret || systemDao?.queryShortcut(word, locale)?.isNotEmpty() ?: false
+        }
+        return ret
+    }
+
     @Synchronized
     fun florisUserDictionaryDao(): UserDictionaryDao? {
-        return if (prefs.suggestion.enabled && prefs.dictionary.enableFlorisUserDictionary) {
+        return if (prefs.dictionary.enableFlorisUserDictionary) {
             florisUserDictionaryDatabase?.userDictionaryDao()
         } else {
             null
@@ -126,7 +170,7 @@ class DictionaryManager private constructor(
 
     @Synchronized
     fun florisUserDictionaryDatabase(): FlorisUserDictionaryDatabase? {
-        return if (prefs.suggestion.enabled && prefs.dictionary.enableFlorisUserDictionary) {
+        return if (prefs.dictionary.enableFlorisUserDictionary) {
             florisUserDictionaryDatabase
         } else {
             null
@@ -135,7 +179,7 @@ class DictionaryManager private constructor(
 
     @Synchronized
     fun systemUserDictionaryDao(): UserDictionaryDao? {
-        return if (prefs.suggestion.enabled && prefs.dictionary.enableSystemUserDictionary) {
+        return if (prefs.dictionary.enableSystemUserDictionary) {
             systemUserDictionaryDatabase?.userDictionaryDao()
         } else {
             null
@@ -144,7 +188,7 @@ class DictionaryManager private constructor(
 
     @Synchronized
     fun systemUserDictionaryDatabase(): SystemUserDictionaryDatabase? {
-        return if (prefs.suggestion.enabled && prefs.dictionary.enableSystemUserDictionary) {
+        return if (prefs.dictionary.enableSystemUserDictionary) {
             systemUserDictionaryDatabase
         } else {
             null
@@ -155,17 +199,15 @@ class DictionaryManager private constructor(
     fun loadUserDictionariesIfNecessary() {
         val context = applicationContext.get() ?: return
 
-        if (prefs.suggestion.enabled) {
-            if (florisUserDictionaryDatabase == null && prefs.dictionary.enableFlorisUserDictionary) {
-                florisUserDictionaryDatabase = Room.databaseBuilder(
-                    context,
-                    FlorisUserDictionaryDatabase::class.java,
-                    FlorisUserDictionaryDatabase.DB_FILE_NAME
-                ).allowMainThreadQueries().build()
-            }
-            if (systemUserDictionaryDatabase == null && prefs.dictionary.enableSystemUserDictionary) {
-                systemUserDictionaryDatabase = SystemUserDictionaryDatabase(context)
-            }
+        if (florisUserDictionaryDatabase == null && prefs.dictionary.enableFlorisUserDictionary) {
+            florisUserDictionaryDatabase = Room.databaseBuilder(
+                context,
+                FlorisUserDictionaryDatabase::class.java,
+                FlorisUserDictionaryDatabase.DB_FILE_NAME
+            ).allowMainThreadQueries().build()
+        }
+        if (systemUserDictionaryDatabase == null && prefs.dictionary.enableSystemUserDictionary) {
+            systemUserDictionaryDatabase = SystemUserDictionaryDatabase(context)
         }
     }
 

app/src/main/java/dev/patrickgold/florisboard/ime/dictionary/Flictionary.kt

@@ -17,11 +17,9 @@
 package dev.patrickgold.florisboard.ime.dictionary
 
 import android.content.Context
-import dev.patrickgold.florisboard.ime.extension.AssetRef
-import dev.patrickgold.florisboard.ime.extension.AssetSource
 import dev.patrickgold.florisboard.ime.nlp.*
+import dev.patrickgold.florisboard.res.FlorisRef
 import java.io.InputStream
-import kotlin.jvm.Throws
 
 /**
  * Class Flictionary which takes care of loading the binary asset as well as providing words for
@@ -30,14 +28,14 @@ import kotlin.jvm.Throws
  * This class accepts binary dictionary files of the type "flict" as defined in here:
  * https://github.com/florisboard/dictionary-tools/blob/main/flictionary.md
  */
-/**
 class Flictionary private constructor(
     override val name: String,
     override val label: String,
     override val authors: List<String>,
     private val date: Long,
     private val version: Int,
-    private val headerStr: String
+    private val headerStr: String,
+    private val languageModel: FlorisLanguageModel
 ) : Dictionary {
     companion object {
         private const val VERSION_0 =                           0x0
@@ -68,11 +66,11 @@ class Flictionary private constructor(
          * either the parsed dictionary or an exception giving information about the error which
          * occurred.
          */
-        fun load(context: Context, assetRef: AssetRef): Result<Flictionary> {
+        fun load(context: Context, assetRef: FlorisRef): Result<Flictionary> {
             val buffer = ByteArray(5000) { 0 }
             val inputStream: InputStream
-            if (assetRef.source == AssetSource.Assets) {
-                inputStream = context.assets.open(assetRef.path)
+            if (assetRef.isAssets) {
+                inputStream = context.assets.open(assetRef.relativePath)
             } else {
                 return Result.failure(Exception("Only AssetSource.Assets is currently supported!"))
             }
@@ -296,26 +294,25 @@ class Flictionary private constructor(
 
     // TODO: preceding tokens are currently ignored
     override fun getTokenPredictions(
-        precedingTokens: List<Token<String>>,
-        currentToken: Token<String>?,
+        precedingTokens: List<Word>,
+        currentToken: Word?,
         maxSuggestionCount: Int,
-        allowPossiblyOffensive: Boolean
-    ): List<WeightedToken<String, Int>> {
-        currentToken ?: return listOf()
+        allowPossiblyOffensive: Boolean,
+        destSuggestionList: SuggestionList
+    ) {
+        currentToken ?: return
 
-        return if (currentToken.data.isNotEmpty()) {
+        if (currentToken.isNotBlank()) {
             val retList = languageModel.matchAllNgrams(
                 ngram = Ngram(
-                    _tokens = listOf(Token(currentToken.data.lowercase())),
+                    _tokens = listOf(Token(currentToken.lowercase())),
                     _freq = -1
                 ),
                 maxEditDistance = 2,
                 maxTokenCount = maxSuggestionCount,
                 allowPossiblyOffensive = allowPossiblyOffensive
             )
-            retList
-        } else {
-            listOf()
+            retList.forEach { destSuggestionList.add(it.data, 128) }
         }
     }
 
@@ -427,4 +424,3 @@ fun InputStream.readNext(b: ByteArray, off: Int, len: Int): Int {
     }
     return lenRead
 }
-*/

app/src/main/java/dev/patrickgold/florisboard/ime/dictionary/UserDictionary.kt

@@ -33,10 +33,9 @@ import androidx.room.RoomDatabase
 import androidx.room.TypeConverter
 import androidx.room.TypeConverters
 import androidx.room.Update
-import dev.patrickgold.florisboard.ime.extension.ExternalContentUtils
-import dev.patrickgold.florisboard.util.LocaleUtils
+import dev.patrickgold.florisboard.common.FlorisLocale
+import dev.patrickgold.florisboard.res.ExternalContentUtils
 import java.lang.ref.WeakReference
-import java.util.*
 
 private const val WORDS_TABLE = "words"
 
@@ -89,28 +88,31 @@ interface UserDictionaryDao {
     fun query(word: String): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.WORD} LIKE '%' || :word || '%' AND $LOCALE_MATCHES")
-    fun query(word: String, locale: Locale?): List<UserDictionaryEntry>
+    fun query(word: String, locale: FlorisLocale?): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.SHORTCUT} = :shortcut")
     fun queryShortcut(shortcut: String): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.SHORTCUT} = :shortcut AND $LOCALE_MATCHES")
-    fun queryShortcut(shortcut: String, locale: Locale?): List<UserDictionaryEntry>
+    fun queryShortcut(shortcut: String, locale: FlorisLocale?): List<UserDictionaryEntry>
 
     @Query(SELECT_ALL_FROM_WORDS)
     fun queryAll(): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE (${UserDictionary.Words.LOCALE} = :locale AND :locale IS NOT NULL) OR (${UserDictionary.Words.LOCALE} IS NULL AND :locale IS NULL)")
-    fun queryAll(locale: Locale?): List<UserDictionaryEntry>
+    fun queryAll(locale: FlorisLocale?): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.WORD} = :word")
     fun queryExact(word: String): List<UserDictionaryEntry>
 
     @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.WORD} = :word AND (${UserDictionary.Words.LOCALE} = :locale OR (${UserDictionary.Words.LOCALE} IS NULL AND :locale IS NULL))")
-    fun queryExact(word: String, locale: Locale?): List<UserDictionaryEntry>
+    fun queryExact(word: String, locale: FlorisLocale?): List<UserDictionaryEntry>
+
+    @Query("$SELECT_ALL_FROM_WORDS WHERE ${UserDictionary.Words.WORD} = :word AND $LOCALE_MATCHES")
+    fun queryExactFuzzyLocale(word: String, locale: FlorisLocale?): List<UserDictionaryEntry>
 
     @Query("SELECT DISTINCT ${UserDictionary.Words.LOCALE} FROM $WORDS_TABLE")
-    fun queryLanguageList(): List<Locale?>
+    fun queryLanguageList(): List<FlorisLocale?>
 
     @Insert
     fun insert(entry: UserDictionaryEntry)
@@ -131,7 +133,7 @@ interface UserDictionaryDatabase {
     fun reset()
 
     fun importCombinedList(context: Context, uri: Uri): Result<Unit> {
-        return ExternalContentUtils.readFromUri(context, uri,6_192_000) { src ->
+        return ExternalContentUtils.readTextFromUri(context, uri,6_192_000) { src ->
             var isFirstLine = true
             src.forEachLine { line ->
                 if (isFirstLine) {
@@ -162,7 +164,7 @@ interface UserDictionaryDatabase {
                     }
                     if (word != null && freq != null) {
                         val alreadyExistingEntries = userDictionaryDao().queryExact(
-                            word!!, locale?.let { LocaleUtils.stringToLocale(it) }
+                            word!!, locale?.let { FlorisLocale.fromTag(it) }
                         )
                         if (alreadyExistingEntries.isNotEmpty()) {
                             userDictionaryDao().update(UserDictionaryEntry(alreadyExistingEntries[0].id, word!!, freq!!, locale, shortcut))
@@ -176,7 +178,7 @@ interface UserDictionaryDatabase {
     }
 
     fun exportCombinedList(context: Context, uri: Uri): Result<Unit> {
-        return ExternalContentUtils.writeToUri(context, uri) { dst ->
+        return ExternalContentUtils.writeTextToUri(context, uri) { dst ->
             StringBuilder().apply {
                 append("dictionary=")
                 append(uri.lastPathSegment)
@@ -223,18 +225,18 @@ abstract class FlorisUserDictionaryDatabase : RoomDatabase(), UserDictionaryData
 
     class Converters {
         @TypeConverter
-        fun localeToString(locale: Locale?): String? {
+        fun localeToString(locale: FlorisLocale?): String? {
             return when (locale) {
                 null -> null
-                else -> locale.toString()
+                else -> locale.localeTag()
             }
         }
 
         @TypeConverter
-        fun stringToLocale(string: String?): Locale? {
+        fun stringToLocale(string: String?): FlorisLocale? {
             return when (string) {
                 null, "all", "null", "" -> null
-                else -> LocaleUtils.stringToLocale(string)
+                else -> FlorisLocale.fromTag(string)
             }
         }
     }
@@ -252,7 +254,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             )
         }
 
-        override fun query(word: String, locale: Locale?): List<UserDictionaryEntry> {
+        override fun query(word: String, locale: FlorisLocale?): List<UserDictionaryEntry> {
             return if (locale == null) {
                 queryResolver(
                     selection = "${UserDictionary.Words.WORD} LIKE ? AND ${UserDictionary.Words.LOCALE} IS NULL",
@@ -262,7 +264,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             } else {
                 queryResolver(
                     selection = "${UserDictionary.Words.WORD} LIKE ? AND (${UserDictionary.Words.LOCALE} = ? OR ${UserDictionary.Words.LOCALE} = ? OR ${UserDictionary.Words.LOCALE} IS NULL)",
-                    selectionArgs = arrayOf("%$word%", locale.toString(), locale.language.toString()),
+                    selectionArgs = arrayOf("%$word%", locale.localeTag(), locale.language),
                     sortOrder = SORT_BY_FREQ_DESC,
                 )
             }
@@ -276,7 +278,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             )
         }
 
-        override fun queryShortcut(shortcut: String, locale: Locale?): List<UserDictionaryEntry> {
+        override fun queryShortcut(shortcut: String, locale: FlorisLocale?): List<UserDictionaryEntry> {
             return if (locale == null) {
                 queryResolver(
                     selection = "${UserDictionary.Words.SHORTCUT} = ? AND ${UserDictionary.Words.LOCALE} IS NULL",
@@ -286,7 +288,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             } else {
                 queryResolver(
                     selection = "${UserDictionary.Words.SHORTCUT} = ? AND (${UserDictionary.Words.LOCALE} = ? OR ${UserDictionary.Words.LOCALE} = ? OR ${UserDictionary.Words.LOCALE} IS NULL)",
-                    selectionArgs = arrayOf(shortcut, locale.toString(), locale.language.toString()),
+                    selectionArgs = arrayOf(shortcut, locale.localeTag(), locale.language),
                     sortOrder = SORT_BY_FREQ_DESC,
                 )
             }
@@ -300,7 +302,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             )
         }
 
-        override fun queryAll(locale: Locale?): List<UserDictionaryEntry> {
+        override fun queryAll(locale: FlorisLocale?): List<UserDictionaryEntry> {
             return if (locale == null) {
                 queryResolver(
                     selection = "${UserDictionary.Words.LOCALE} IS NULL",
@@ -310,7 +312,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             } else {
                 queryResolver(
                     selection = "${UserDictionary.Words.LOCALE} = ?",
-                    selectionArgs = arrayOf(locale.toString()),
+                    selectionArgs = arrayOf(locale.localeTag()),
                     sortOrder = SORT_BY_FREQ_DESC,
                 )
             }
@@ -324,7 +326,7 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
             )
         }
 
-        override fun queryExact(word: String, locale: Locale?): List<UserDictionaryEntry> {
+        override fun queryExact(word: String, locale: FlorisLocale?): List<UserDictionaryEntry> {
             return if (locale == null) {
                 queryResolver(
                     selection = "${UserDictionary.Words.WORD} = ? AND ${UserDictionary.Words.LOCALE} IS NULL",
@@ -333,14 +335,30 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
                 )
             } else {
                 queryResolver(
-                    selection = "${UserDictionary.Words.WORD} LIKE ? AND ${UserDictionary.Words.LOCALE} = ?",
-                    selectionArgs = arrayOf(word, locale.toString()),
+                    selection = "${UserDictionary.Words.WORD} = ? AND ${UserDictionary.Words.LOCALE} = ?",
+                    selectionArgs = arrayOf(word, locale.localeTag()),
                     sortOrder = SORT_BY_FREQ_DESC,
                 )
             }
         }
 
-        override fun queryLanguageList(): List<Locale?> {
+        override fun queryExactFuzzyLocale(word: String, locale: FlorisLocale?): List<UserDictionaryEntry> {
+            return if (locale == null) {
+                queryResolver(
+                    selection = "${UserDictionary.Words.WORD} = ? AND ${UserDictionary.Words.LOCALE} IS NULL",
+                    selectionArgs = arrayOf(word),
+                    sortOrder = SORT_BY_FREQ_DESC,
+                )
+            } else {
+                queryResolver(
+                    selection = "${UserDictionary.Words.WORD} = ? AND (${UserDictionary.Words.LOCALE} = ? OR ${UserDictionary.Words.LOCALE} IS NULL)",
+                    selectionArgs = arrayOf(word, locale.localeTag()),
+                    sortOrder = SORT_BY_FREQ_DESC,
+                )
+            }
+        }
+
+        override fun queryLanguageList(): List<FlorisLocale?> {
             val resolver = applicationContext.get()?.contentResolver ?: return listOf()
             val cursor = resolver.query(
                 UserDictionary.Words.CONTENT_URI,
@@ -353,13 +371,13 @@ class SystemUserDictionaryDatabase(context: Context) : UserDictionaryDatabase {
                 return listOf()
             }
             val localeIndex = cursor.getColumnIndex(UserDictionary.Words.LOCALE)
-            val retList = mutableSetOf<Locale?>()
+            val retList = mutableSetOf<FlorisLocale?>()
             while (cursor.moveToNext()) {
                 val localeStr = cursor.getString(localeIndex)
                 if (localeStr == null) {
                     retList.add(null)
                 } else {
-                    retList.add(LocaleUtils.stringToLocale(localeStr))
+                    retList.add(FlorisLocale.fromTag(localeStr))
                 }
             }
             cursor.close()

app/src/main/java/dev/patrickgold/florisboard/ime/dictionary/temp_suggestion_code.kt

@@ -0,0 +1,441 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.ime.dictionary
+
+import dev.patrickgold.florisboard.ime.nlp.SuggestionList
+
+/*
+ * ====================== IMPORTANT ========================
+ *
+ * All code in this file is only temporary added back in so the stable track has suggestions again.
+ * In 0.3.15 a renewed suggestion algorithm will be built and this mess will be removed!
+ *
+ * ==========================================================
+ */
+
+/**
+ * Abstract interface representing a n-gram of tokens. Each n-gram instance can be assigned a
+ * unique frequency [freq].
+ */
+open class Ngram<T : Any, F : Comparable<F>>(_tokens: List<Token<T>>, _freq: F) {
+    companion object {
+        /** Constant order value for unigrams. */
+        const val ORDER_UNIGRAM: Int = 1
+
+        /** Constant order value for bigrams. */
+        const val ORDER_BIGRAM: Int = 2
+
+        /** Constant order value for trigrams. */
+        const val ORDER_TRIGRAM: Int = 3
+    }
+
+    init {
+        if (_tokens.size < ORDER_UNIGRAM) {
+            throw Exception("A n-gram must contain at least 1 token!")
+        }
+    }
+
+    /**
+     * A list of tokens for this n-gram. The length of this list is guaranteed to be matching
+     * [order].
+     */
+    val tokens: List<Token<T>> = _tokens
+
+    /**
+     * The frequency value of this n-gram.
+     */
+    val freq: F = _freq
+
+    /**
+     * The order of this n-gram (1, 2, 3, ...).
+     */
+    val order: Int
+        get() = tokens.size
+}
+
+/**
+ * Abstract interface representing a token used in [Ngram].
+ */
+open class Token<T : Any>(_data: T) {
+    /**
+     * The data of this token.
+     */
+    val data: T = _data
+
+    override fun toString(): String {
+        return "Token(\"$data\")"
+    }
+
+    override fun hashCode(): Int {
+        return data.hashCode()
+    }
+
+    override fun equals(other: Any?): Boolean {
+        if (this === other) return true
+        if (javaClass != other?.javaClass) return false
+
+        other as Token<*>
+
+        if (data != other.data) return false
+
+        return true
+    }
+}
+
+/**
+ * Converts a list of tokens carrying [CharSequence] data to a list of [CharSequence].
+ */
+fun List<Token<CharSequence>>.toCharSequenceList(): List<CharSequence> {
+    return this.map { it.data }
+}
+
+/**
+ * Converts a list of tokens carrying [String] data to a list of [String].
+ */
+fun List<Token<String>>.toStringList(): List<String> {
+    return this.map { it.data }
+}
+
+/**
+ * Abstract interface for a language model. Can house any n-grams with a minimum order of one.
+ */
+interface LanguageModel<T : Any, F : Comparable<F>> {
+    /**
+     * Tries to get the n-gram for the passed [tokens]. Throws a NPE if no match could be found.
+     */
+    @Throws(NullPointerException::class)
+    fun getNgram(vararg tokens: T): Ngram<T, F>
+
+    /**
+     * Tries to get the n-gram for the passed [ngram], whereas the frequency is ignored while
+     * searching. Throws a NPE if no match could be found.
+     */
+    @Throws(NullPointerException::class)
+    fun getNgram(ngram: Ngram<T, F>): Ngram<T, F>
+
+    /**
+     * Tries to get the n-gram for the passed [tokens]. Returns null if no match could be found.
+     */
+    fun getNgramOrNull(vararg tokens: T): Ngram<T, F>?
+
+    /**
+     * Tries to get the n-gram for the passed [ngram], whereas the frequency is ignored while
+     * searching. Returns null if no match could be found.
+     */
+    fun getNgramOrNull(ngram: Ngram<T, F>): Ngram<T, F>?
+
+    /**
+     * Checks if a given [ngram] exists within this model. If [doMatchFreq] is set to true, the
+     * frequency is also matched.
+     */
+    fun hasNgram(ngram: Ngram<T, F>, doMatchFreq: Boolean = false): Boolean
+
+    /**
+     * Matches all n-grams which match the given [ngram], whereas the last item in the n-gram is
+     * is used to search for predictions.
+     */
+    fun matchAllNgrams(
+        ngram: Ngram<T, F>,
+        maxEditDistance: Int,
+        maxTokenCount: Int,
+        allowPossiblyOffensive: Boolean
+    ): List<Token<T>>
+}
+
+/**
+ * Mutable version of [LanguageModel].
+ */
+interface MutableLanguageModel<T : Any, F : Comparable<F>> : LanguageModel<T, F> {
+    fun deleteNgram(ngram: Ngram<T, F>)
+
+    fun insertNgram(ngram: Ngram<T, F>)
+
+    fun updateNgram(ngram: Ngram<T, F>)
+}
+
+/**
+ * Represents the root node to a n-gram tree.
+ */
+open class NgramTree(
+    sameOrderChildren: MutableList<NgramNode> = mutableListOf(),
+    higherOrderChildren: MutableList<NgramNode> = mutableListOf()
+) : NgramNode(0, '?', -1, sameOrderChildren, higherOrderChildren)
+
+/**
+ * A node of a n-gram tree, which holds the character it represents, the corresponding frequency,
+ * a pre-computed string representing all parent characters and the current one as well as child
+ * nodes, one for the same order n-gram nodes and one for the higher order n-gram nodes.
+ */
+open class NgramNode(
+    val order: Int,
+    val char: Char,
+    val freq: Int,
+    val sameOrderChildren: MutableList<NgramNode> = mutableListOf(),
+    val higherOrderChildren: MutableList<NgramNode> = mutableListOf()
+) {
+    companion object {
+        const val FREQ_CHARACTER = -1
+        const val FREQ_WORD_MIN = 0
+        const val FREQ_WORD_MAX = 255
+        const val FREQ_WORD_FILLER = -2
+        const val FREQ_IS_POSSIBLY_OFFENSIVE = 0
+    }
+
+    val isCharacter: Boolean
+        get() = freq == FREQ_CHARACTER
+
+    val isWord: Boolean
+        get() = freq in FREQ_WORD_MIN..FREQ_WORD_MAX
+
+    val isWordFiller: Boolean
+        get() = freq == FREQ_WORD_FILLER
+
+    val isPossiblyOffensive: Boolean
+        get() = freq == FREQ_IS_POSSIBLY_OFFENSIVE
+
+    fun findWord(word: String): NgramNode? {
+        var currentNode = this
+        for ((pos, char) in word.withIndex()) {
+            val childNode = if (pos == 0) {
+                currentNode.higherOrderChildren.find { it.char == char }
+            } else {
+                currentNode.sameOrderChildren.find { it.char == char }
+            }
+            if (childNode != null) {
+                currentNode = childNode
+            } else {
+                return null
+            }
+        }
+        return if (currentNode.isWord || currentNode.isWordFiller) {
+            currentNode
+        } else {
+            null
+        }
+    }
+
+    /**
+     * This function allows to search for a given [input] word with a given [maxEditDistance] and
+     * adds all matches in the trie to the [list].
+     */
+    fun listSimilarWords(
+        input: String,
+        list: SuggestionList,
+        word: StringBuilder,
+        allowPossiblyOffensive: Boolean,
+        maxEditDistance: Int,
+        deletionCost: Int = 0,
+        insertionCost: Int = 0,
+        substitutionCost: Int = 0,
+        pos: Int = -1
+    ) {
+        if (pos > -1) {
+            word.append(char)
+        }
+        val costSum = deletionCost + insertionCost + substitutionCost
+        if (pos > -1 && (pos + 1 == input.length) && isWord && ((isPossiblyOffensive && allowPossiblyOffensive)
+                || !isPossiblyOffensive)) {
+            // Using shift right instead of divide by 2^(costSum) as it is mathematically the
+            // same but faster.
+            list.add(word.toString(), freq shr costSum)
+        }
+        if (pos <= -1) {
+            for (childNode in higherOrderChildren) {
+                childNode.listSimilarWords(
+                    input, list, word, allowPossiblyOffensive, maxEditDistance, 0, 0, 0, 0
+                )
+            }
+        } else if (maxEditDistance == costSum) {
+            if (pos + 1 < input.length) {
+                sameOrderChildren.find { it.char == input[pos + 1] }?.listSimilarWords(
+                    input, list, word, allowPossiblyOffensive, maxEditDistance,
+                    deletionCost, insertionCost, substitutionCost, pos + 1
+                )
+            }
+        } else {
+            // Delete
+            if (pos + 2 < input.length) {
+                sameOrderChildren.find { it.char == input[pos + 2] }?.listSimilarWords(
+                    input, list, word, allowPossiblyOffensive, maxEditDistance,
+                    deletionCost + 1, insertionCost, substitutionCost, pos + 2
+                )
+            }
+            for (childNode in sameOrderChildren) {
+                if (pos + 1 < input.length && childNode.char == input[pos + 1]) {
+                    childNode.listSimilarWords(
+                        input, list, word, allowPossiblyOffensive, maxEditDistance,
+                        deletionCost, insertionCost, substitutionCost, pos + 1
+                    )
+                } else  {
+                    // Insert
+                    childNode.listSimilarWords(
+                        input, list, word, allowPossiblyOffensive, maxEditDistance,
+                        deletionCost, insertionCost + 1, substitutionCost, pos
+                    )
+                    if (pos + 1 < input.length) {
+                        // Substitute
+                        childNode.listSimilarWords(
+                            input, list, word, allowPossiblyOffensive, maxEditDistance,
+                            deletionCost, insertionCost, substitutionCost + 1, pos + 1
+                        )
+                    }
+                }
+            }
+        }
+        if (pos > -1) {
+            word.deleteAt(word.lastIndex)
+        }
+    }
+
+    fun listAllSameOrderWords(list: SuggestionList, word: StringBuilder, allowPossiblyOffensive: Boolean) {
+        word.append(char)
+        if (isWord && ((isPossiblyOffensive && allowPossiblyOffensive) || !isPossiblyOffensive)) {
+            list.add(word.toString(), freq)
+        }
+        for (childNode in sameOrderChildren) {
+            childNode.listAllSameOrderWords(list, word, allowPossiblyOffensive)
+        }
+        word.deleteAt(word.lastIndex)
+    }
+}
+
+open class FlorisLanguageModel(
+    initTreeObj: NgramTree? = null
+) : LanguageModel<String, Int> {
+    protected val ngramTree: NgramTree = initTreeObj ?: NgramTree()
+
+    override fun getNgram(vararg tokens: String): Ngram<String, Int> {
+        val ngramOut = getNgramOrNull(*tokens)
+        if (ngramOut != null) {
+            return ngramOut
+        } else {
+            throw NullPointerException("No n-gram found matching the given tokens: $tokens")
+        }
+    }
+
+    override fun getNgram(ngram: Ngram<String, Int>): Ngram<String, Int> {
+        val ngramOut = getNgramOrNull(ngram)
+        if (ngramOut != null) {
+            return ngramOut
+        } else {
+            throw NullPointerException("No n-gram found matching the given ngram: $ngram")
+        }
+    }
+
+    override fun getNgramOrNull(vararg tokens: String): Ngram<String, Int>? {
+        var currentNode: NgramNode = ngramTree
+        for (token in tokens) {
+            val childNode = currentNode.findWord(token)
+            if (childNode != null) {
+                currentNode = childNode
+            } else {
+                return null
+            }
+        }
+        return Ngram(tokens.toList().map { Token(it) }, currentNode.freq)
+    }
+
+    override fun getNgramOrNull(ngram: Ngram<String, Int>): Ngram<String, Int>? {
+        return getNgramOrNull(*ngram.tokens.toStringList().toTypedArray())
+    }
+
+    override fun hasNgram(ngram: Ngram<String, Int>, doMatchFreq: Boolean): Boolean {
+        val result = getNgramOrNull(ngram)
+        return if (result != null) {
+            if (doMatchFreq) {
+                ngram.freq == result.freq
+            } else {
+                true
+            }
+        } else {
+            false
+        }
+    }
+
+    override fun matchAllNgrams(
+        ngram: Ngram<String, Int>,
+        maxEditDistance: Int,
+        maxTokenCount: Int,
+        allowPossiblyOffensive: Boolean
+    ): List<Token<String>> {
+        val ngramList = mutableListOf<Token<String>>()
+        var currentNode: NgramNode = ngramTree
+        for ((t, token) in ngram.tokens.withIndex()) {
+            val word = token.data
+            if (t + 1 >= ngram.tokens.size) {
+                if (word.isNotEmpty()) {
+                    // The last word is not complete, so find all possible words and sort
+                    val splitWord = mutableListOf<Char>()
+                    var splitNode: NgramNode? = currentNode
+                    for ((pos, char) in word.withIndex()) {
+                        val node = if (pos == 0) {
+                            splitNode?.higherOrderChildren?.find { it.char == char }
+                        } else {
+                            splitNode?.sameOrderChildren?.find { it.char == char }
+                        }
+                        splitWord.add(char)
+                        splitNode = node
+                        if (node == null) {
+                            break
+                        }
+                    }
+                    if (splitNode != null) {
+                        // Input thus far is valid
+                        val wordNodes = SuggestionList.new(maxTokenCount)
+                        val strBuilder = StringBuilder().append(word.substring(0, word.length - 1))
+                        splitNode.listAllSameOrderWords(wordNodes, strBuilder, allowPossiblyOffensive)
+                        ngramList.addAll(wordNodes.map { Token(it) })
+                    }
+                    if (ngramList.size < maxTokenCount) {
+                        val wordNodes = SuggestionList.new(maxTokenCount)
+                        val strBuilder = StringBuilder()
+                        currentNode.listSimilarWords(word, wordNodes, strBuilder, allowPossiblyOffensive, maxEditDistance)
+                        ngramList.addAll(wordNodes.map { Token(it) })
+                    }
+                }
+            } else {
+                val node = currentNode.findWord(word)
+                if (node == null) {
+                    return ngramList
+                } else {
+                    currentNode = node
+                }
+            }
+        }
+        return ngramList
+    }
+
+    fun toFlorisMutableLanguageModel(): FlorisMutableLanguageModel = FlorisMutableLanguageModel(ngramTree)
+}
+
+open class FlorisMutableLanguageModel(
+    initTreeObj: NgramTree? = null
+) : MutableLanguageModel<String, Int>, FlorisLanguageModel(initTreeObj) {
+    override fun deleteNgram(ngram: Ngram<String, Int>) {
+        TODO("Not yet implemented")
+    }
+
+    override fun insertNgram(ngram: Ngram<String, Int>) {
+        TODO("Not yet implemented")
+    }
+
+    override fun updateNgram(ngram: Ngram<String, Int>) {
+        TODO("Not yet implemented")
+    }
+
+    fun toFlorisLanguageModel(): FlorisLanguageModel = FlorisLanguageModel(ngramTree)
+}

app/src/main/java/dev/patrickgold/florisboard/ime/extension/AssetManager.kt

@@ -1,268 +0,0 @@
-/*
- * Copyright (C) 2020 Patrick Goldinger
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package dev.patrickgold.florisboard.ime.extension
-
-import android.content.Context
-import android.net.Uri
-import dev.patrickgold.florisboard.ime.keyboard.CaseSelector
-import dev.patrickgold.florisboard.ime.keyboard.KeyData
-import dev.patrickgold.florisboard.ime.keyboard.VariationSelector
-import dev.patrickgold.florisboard.ime.media.emoji.EmojiKeyData
-import dev.patrickgold.florisboard.ime.text.composing.*
-import dev.patrickgold.florisboard.ime.text.keyboard.*
-import kotlinx.serialization.decodeFromString
-import kotlinx.serialization.encodeToString
-import kotlinx.serialization.json.Json
-import kotlinx.serialization.modules.SerializersModule
-import kotlinx.serialization.modules.polymorphic
-import timber.log.Timber
-import java.io.File
-
-class AssetManager private constructor(val applicationContext: Context) {
-    private val json = Json {
-        classDiscriminator = "$"
-        ignoreUnknownKeys = true
-        isLenient = true
-        serializersModule = SerializersModule {
-            polymorphic(KeyData::class) {
-                subclass(BasicTextKeyData::class, BasicTextKeyData.serializer())
-                subclass(AutoTextKeyData::class, AutoTextKeyData.serializer())
-                subclass(MultiTextKeyData::class, MultiTextKeyData.serializer())
-                subclass(EmojiKeyData::class, EmojiKeyData.serializer())
-                subclass(CaseSelector::class, CaseSelector.serializer())
-                subclass(VariationSelector::class, VariationSelector.serializer())
-                default { BasicTextKeyData.serializer() }
-            }
-            polymorphic(TextKeyData::class) {
-                subclass(BasicTextKeyData::class, BasicTextKeyData.serializer())
-                subclass(AutoTextKeyData::class, AutoTextKeyData.serializer())
-                subclass(MultiTextKeyData::class, MultiTextKeyData.serializer())
-                default { BasicTextKeyData.serializer() }
-            }
-            polymorphic(Composer::class) {
-                subclass(Appender::class, Appender.serializer())
-                subclass(HangulUnicode::class, HangulUnicode.serializer())
-                subclass(WithRules::class, WithRules.serializer())
-                default { Appender.serializer() }
-            }
-        }
-    }
-
-    companion object {
-        private var defaultInstance: AssetManager? = null
-
-        fun init(applicationContext: Context): AssetManager {
-            val instance = AssetManager(applicationContext)
-            defaultInstance = instance
-            return instance
-        }
-
-        fun default(): AssetManager {
-            val instance = defaultInstance
-            if (instance != null) {
-                return instance
-            } else {
-                throw UninitializedPropertyAccessException(
-                    "${this::class.simpleName} has not been initialized previously. Make sure to call init(applicationContext) before using default()."
-                )
-            }
-        }
-
-        fun defaultOrNull(): AssetManager? = defaultInstance
-    }
-
-    fun jsonBuilder(): Json = json
-
-    fun deleteAsset(ref: AssetRef): Result<Unit> {
-        return when (ref.source) {
-            AssetSource.Internal -> {
-                val file = File(applicationContext.filesDir.absolutePath + "/" + ref.path)
-                if (file.isFile) {
-                    val success = file.delete()
-                    if (success) {
-                        Result.success(Unit)
-                    } else {
-                        Result.failure(Exception("Could not delete file."))
-                    }
-                } else {
-                    Result.failure(Exception("Provided reference is not a file."))
-                }
-            }
-            else -> Result.failure(Exception("Can not delete an asset in source '${ref.source}'"))
-        }
-    }
-
-    fun hasAsset(ref: AssetRef): Boolean {
-        return when (ref.source) {
-            AssetSource.Assets -> {
-                try {
-                    val file = File(ref.path)
-                    val list = applicationContext.assets.list(file.parent?.toString() ?: "")
-                    list?.contains(file.name) == true
-                } catch (e: Exception) {
-                    false
-                }
-            }
-            AssetSource.Internal -> {
-                val file = File(applicationContext.filesDir.absolutePath + "/" + ref.path)
-                file.exists() && file.isFile
-            }
-            else -> false
-        }
-    }
-
-    inline fun <reified T> listAssets(ref: AssetRef): Result<Map<AssetRef, T>> {
-        val retMap = mutableMapOf<AssetRef, T>()
-        return when (ref.source) {
-            AssetSource.Assets -> runCatching {
-                val list = applicationContext.assets.list(ref.path)
-                if (list != null) {
-                    for (file in list) {
-                        val fileRef = ref.copy(path = ref.path + "/" + file)
-                        val assetResult = loadJsonAsset<T>(fileRef)
-                        assetResult.onSuccess { asset ->
-                            retMap[fileRef.copy()] = asset
-                        }.onFailure { error ->
-                            Timber.e(error.toString())
-                        }
-                    }
-                }
-                retMap.toMap()
-            }
-            AssetSource.Internal -> {
-                val dir = File(applicationContext.filesDir.absolutePath + "/" + ref.path)
-                if (dir.isDirectory) {
-                    dir.listFiles()?.let {
-                        it.forEach { file ->
-                            if (file.isFile) {
-                                val fileRef = ref.copy(path = ref.path + "/" + file.name)
-                                val assetResult = loadJsonAsset<T>(fileRef)
-                                assetResult.onSuccess { asset ->
-                                    retMap[fileRef.copy()] = asset
-                                }.onFailure { error ->
-                                    Timber.e(error.toString())
-                                }
-                            }
-                        }
-                    }
-                }
-                Result.success(retMap.toMap())
-            }
-            else -> Result.success(retMap.toMap())
-        }
-    }
-
-    inline fun <reified T> loadJsonAsset(ref: AssetRef): Result<T> {
-        return loadTextAsset(ref).fold(
-            onSuccess = { runCatching { jsonBuilder().decodeFromString(it) } },
-            onFailure = { Result.failure(it) }
-        )
-    }
-
-    inline fun <reified T> loadJsonAsset(uri: Uri, maxSize: Int): Result<T> {
-        return loadTextAsset(uri, maxSize).fold(
-            onSuccess = { runCatching { jsonBuilder().decodeFromString(it) } },
-            onFailure = { Result.failure(it) }
-        )
-    }
-
-    fun loadTextAsset(ref: AssetRef): Result<String> {
-        return when (ref.source) {
-            is AssetSource.Assets -> runCatching {
-                applicationContext.assets.open(ref.path).bufferedReader().use { it.readText() }
-            }
-            is AssetSource.Internal -> {
-                val file = File(applicationContext.filesDir.absolutePath + "/" + ref.path)
-                val contents = readTextFile(file).getOrElse { return Result.failure(it) }
-                if (contents.isBlank()) {
-                    Result.failure(Exception("File is blank!"))
-                } else {
-                    Result.success(contents)
-                }
-            }
-            else -> Result.failure(Exception("Unsupported asset ref!"))
-        }
-    }
-
-    /**
-     * Reads a given [file] and returns its content.
-     *
-     * @param file The file object.
-     * @return The contents of the file or an empty string, if the file does not exist.
-     */
-    private fun readTextFile(file: File) = runCatching {
-        val retText = StringBuilder()
-        if (file.exists()) {
-            val newLine = System.lineSeparator()
-            file.forEachLine {
-                retText.append(it)
-                retText.append(newLine)
-            }
-        }
-        retText.toString()
-    }
-
-    fun loadTextAsset(uri: Uri, maxSize: Int): Result<String> {
-        return ExternalContentUtils.readTextFromUri(applicationContext, uri, maxSize)
-    }
-
-    inline fun <reified T> writeJsonAsset(ref: AssetRef, asset: T): Result<Unit> {
-        return runCatching { jsonBuilder().encodeToString(asset) }.fold(
-            onSuccess = { writeTextAsset(ref, it) },
-            onFailure = { Result.failure(it) }
-        )
-    }
-
-    inline fun <reified T> writeJsonAsset(uri: Uri, asset: T): Result<Unit> {
-        return runCatching { jsonBuilder().encodeToString(asset) }.fold(
-            onSuccess = { writeTextAsset(uri, it) },
-            onFailure = { Result.failure(it) }
-        )
-    }
-
-    fun writeTextAsset(ref: AssetRef, text: String): Result<Unit> {
-        return when (ref.source) {
-            AssetSource.Internal -> {
-                val file = File(applicationContext.filesDir.absolutePath + "/" + ref.path)
-                writeTextFile(file, text)
-            }
-            else -> Result.failure(Exception("Can not write an asset in source '${ref.source}'"))
-        }
-    }
-
-    /**
-     * Writes given [text] to given [file]. If the file already exists, its current content
-     * will be overwritten.
-     *
-     * @param file The file object.
-     * @param text The text to write to the file.
-     * @return The contents of the file or an empty string, if the file does not exist.
-     */
-    private fun writeTextFile(file: File, text: String) = runCatching {
-        file.parent?.let {
-            val dir = File(it)
-            if (!dir.exists()) {
-                dir.mkdirs()
-            }
-        }
-        file.writeText(text)
-    }
-
-    fun writeTextAsset(uri: Uri, text: String): Result<Unit> {
-        return ExternalContentUtils.writeTextToUri(applicationContext, uri, text)
-    }
-}

app/src/main/java/dev/patrickgold/florisboard/ime/extension/AssetRef.kt

@@ -1,54 +0,0 @@
-/*
- * Copyright (C) 2020 Patrick Goldinger
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package dev.patrickgold.florisboard.ime.extension
-
-/**
- * Data class which is a reference to an asset file. It indicates in which storage medium the asset
- * is as well as the relative path to it.
- *
- * @property source The source in which the asset is (APK assets, internal storage, external)
- * @property path The relative path to the asset within [source]. Must not begin and end with a
- *  forward slash.
- */
-data class AssetRef(
-    val source: AssetSource,
-    val path: String
-) {
-    companion object {
-        private const val DELIMITER: String = ":"
-
-        fun fromString(str: String): Result<AssetRef> {
-            val items = str.split(DELIMITER)
-            if (items.size != 2) {
-                return Result.failure(Exception("Unexpected length of given asset ref. Make sure that the asset ref string contains exactly 2 items separated by '$DELIMITER'!"))
-            }
-            val retSource = AssetSource.fromString(items[0]).getOrElse {
-                return Result.failure(Exception(it))
-            }
-            return Result.success(AssetRef(retSource, items[1]))
-        }
-    }
-
-    override fun toString(): String {
-        val retString: StringBuilder = StringBuilder().apply {
-            append(source.toString())
-            append(DELIMITER)
-            append(path)
-        }
-        return retString.toString()
-    }
-}

app/src/main/java/dev/patrickgold/florisboard/ime/extension/AssetSource.kt

@@ -1,73 +0,0 @@
-/*
- * Copyright (C) 2020 Patrick Goldinger
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package dev.patrickgold.florisboard.ime.extension
-
-/**
- * Sealed class which specifies where an asset comes from. There are 3 different types, all of which
- * require a different approach on how to access the actual asset.
- */
-sealed class AssetSource {
-    /**
-     * The asset comes pre-built with the application, thus all paths must be relative to the asset
-     * directory of FlorisBoard.
-     */
-    object Assets : AssetSource()
-
-    /**
-     * The asset is saved in the internal storage of FlorisBoard, all relative paths must therefore
-     * be treated as such.
-     */
-    object Internal : AssetSource()
-
-    /**
-     * Asset source is an external extension, which requires the package name and possibly other
-     * data. Currently NYI.
-     * TODO: Implement external extensions
-     */
-    data class External(val packageName: String) : AssetSource() {
-        override fun toString(): String {
-            return super.toString()
-        }
-    }
-
-    companion object {
-        private val externalRegex: Regex = """^external\\(([a-z]+\\.)*[a-z]+\\)\$""".toRegex()
-
-        fun fromString(str: String): Result<AssetSource> {
-            return when (val string = str.lowercase()) {
-                "assets" -> Result.success(Assets)
-                "internal" -> Result.success(Internal)
-                else -> {
-                    if (string.matches(externalRegex)) {
-                        val packageName = string.substring(9, string.length - 1)
-                        Result.success(External(packageName))
-                    } else {
-                        Result.failure(Exception("'$str' is not a valid AssetSource."))
-                    }
-                }
-            }
-        }
-    }
-
-    override fun toString(): String {
-        return when (this) {
-            is Assets -> "assets"
-            is Internal -> "internal"
-            is External -> "external($packageName)"
-        }
-    }
-}

app/src/main/java/dev/patrickgold/florisboard/ime/keyboard/ComputingEvaluator.kt

@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.ime.keyboard
+
+import dev.patrickgold.florisboard.ime.core.Subtype
+import dev.patrickgold.florisboard.ime.text.key.*
+
+interface ComputingEvaluator {
+    fun evaluateCaps(): Boolean
+
+    fun evaluateCaps(data: KeyData): Boolean
+
+    fun evaluateCharHalfWidth(): Boolean = false
+
+    fun evaluateKanaKata(): Boolean = false
+
+    fun evaluateKanaSmall(): Boolean = false
+
+    fun evaluateEnabled(data: KeyData): Boolean
+
+    fun evaluateVisible(data: KeyData): Boolean
+
+    fun getActiveSubtype(): Subtype
+
+    fun getKeyVariation(): KeyVariation
+
+    fun getKeyboard(): Keyboard
+
+    fun isSlot(data: KeyData): Boolean
+
+    fun getSlotData(data: KeyData): KeyData?
+}
+
+object DefaultComputingEvaluator : ComputingEvaluator {
+    override fun evaluateCaps(): Boolean = false
+
+    override fun evaluateCaps(data: KeyData): Boolean = false
+
+    override fun evaluateCharHalfWidth(): Boolean = false
+
+    override fun evaluateKanaKata(): Boolean = false
+
+    override fun evaluateKanaSmall(): Boolean = false
+
+    override fun evaluateEnabled(data: KeyData): Boolean = true
+
+    override fun evaluateVisible(data: KeyData): Boolean = true
+
+    override fun getActiveSubtype(): Subtype = Subtype.DEFAULT
+
+    override fun getKeyVariation(): KeyVariation = KeyVariation.NORMAL
+
+    override fun getKeyboard(): Keyboard = throw NotImplementedError()
+
+    override fun isSlot(data: KeyData): Boolean = false
+
+    override fun getSlotData(data: KeyData): KeyData? = null
+}

app/src/main/java/dev/patrickgold/florisboard/ime/keyboard/InputFeedbackManager.kt

@@ -0,0 +1,145 @@
+/*
+ * Copyright (C) 2021 Patrick Goldinger
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.patrickgold.florisboard.ime.keyboard
+
+import android.content.Context
+import android.inputmethodservice.InputMethodService
+import android.media.AudioManager
+import android.os.Build
+import android.os.VibrationEffect
+import android.os.Vibrator
+import android.provider.Settings
+import android.view.HapticFeedbackConstants
+import dev.patrickgold.florisboard.debug.flogDebug
+import dev.patrickgold.florisboard.ime.core.Preferences
+import dev.patrickgold.florisboard.ime.text.key.KeyCode
+import dev.patrickgold.florisboard.ime.text.keyboard.TextKeyData
+
+/**
+ * Input feedback manager responsible to process and perform audio and haptic
+ * feedback for user interactions based on the system and floris preferences.
+ */
+class InputFeedbackManager private constructor(private val ims: InputMethodService) {
+    companion object {
+        fun new(ims: InputMethodService) = InputFeedbackManager(ims)
+    }
+
+    private val prefs get() = Preferences.default()
+
+    private val audioManager = ims.getSystemService(Context.AUDIO_SERVICE) as? AudioManager
+    private val vibrator = ims.getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator
+    private val contentResolver = ims.contentResolver
+
+    fun keyPress(data: KeyData = TextKeyData.UNSPECIFIED) {
+        if (prefs.inputFeedback.audioFeatKeyPress) performAudioFeedback(data, 1.0)
+        if (prefs.inputFeedback.hapticFeatKeyPress) performHapticFeedback(data, 1.0)
+    }
+
+    fun keyLongPress(data: KeyData = TextKeyData.UNSPECIFIED) {
+        if (prefs.inputFeedback.audioFeatKeyLongPress) performAudioFeedback(data, 0.7)
+        if (prefs.inputFeedback.hapticFeatKeyLongPress) performHapticFeedback(data, 0.4)
+    }
+
+    fun keyRepeatedAction(data: KeyData = TextKeyData.UNSPECIFIED) {
+        if (prefs.inputFeedback.audioFeatKeyRepeatedAction) performAudioFeedback(data, 0.4)
+        if (prefs.inputFeedback.hapticFeatKeyRepeatedAction) performHapticFeedback(data, 0.05)
+    }
+
+    fun gestureSwipe(data: KeyData = TextKeyData.UNSPECIFIED) {
+        if (prefs.inputFeedback.audioFeatGestureSwipe) performAudioFeedback(data, 0.7)
+        if (prefs.inputFeedback.hapticFeatGestureSwipe) performHapticFeedback(data, 0.4)
+    }
+
+    fun gestureMovingSwipe(data: KeyData = TextKeyData.UNSPECIFIED) {
+        if (prefs.inputFeedback.audioFeatGestureMovingSwipe) performAudioFeedback(data, 0.4)
+        if (prefs.inputFeedback.hapticFeatGestureMovingSwipe) performHapticFeedback(data, 0.05)
+    }
+
+    private fun systemPref(id: String): Boolean {
+        if (contentResolver == null) return false
+        return Settings.System.getInt(contentResolver, id, 0) != 0
+    }
+
+    private fun performAudioFeedback(data: KeyData, factor: Double) {
+        if (audioManager == null) return
+        if (!prefs.inputFeedback.audioEnabled) return
+
+        if (!prefs.inputFeedback.audioIgnoreSystemSettings) {
+            if (!systemPref(Settings.System.SOUND_EFFECTS_ENABLED)) return
+        }
+
+        val volume = (prefs.inputFeedback.audioVolume * factor) / 100.0
+        val effect = when (data.code) {
+            KeyCode.DELETE -> AudioManager.FX_KEYPRESS_DELETE
+            KeyCode.ENTER -> AudioManager.FX_KEYPRESS_RETURN
+            KeyCode.SPACE -> AudioManager.FX_KEYPRESS_SPACEBAR
+            else -> AudioManager.FX_KEYPRESS_STANDARD
+        }
+        if (volume in 0.01..1.00) {
+            flogDebug { "Perform audio with volume=$volume and effect=$effect" }
+            audioManager.playSoundEffect(effect, volume.toFloat())
+        }
+    }
+
+    private fun performHapticFeedback(data: KeyData, factor: Double) {
+        if (vibrator == null || !vibrator.hasVibrator()) return
+        if (!prefs.inputFeedback.hapticEnabled) return
+
+        if (!prefs.inputFeedback.hapticIgnoreSystemSettings) {
+            if (!systemPref(Settings.System.HAPTIC_FEEDBACK_ENABLED)) return
+        }
+
+        if (!prefs.inputFeedback.hapticUseVibrator) {
+            val view = ims.window?.window?.decorView ?: return
+            val hfc = if (factor < 1.0 && Build.VERSION.SDK_INT >= Build.VERSION_CODES.O_MR1) {
+                HapticFeedbackConstants.TEXT_HANDLE_MOVE
+            } else {
+                HapticFeedbackConstants.KEYBOARD_TAP
+            }
+            val didPerform = view.performHapticFeedback(hfc,
+                HapticFeedbackConstants.FLAG_IGNORE_VIEW_SETTING or
+                    HapticFeedbackConstants.FLAG_IGNORE_GLOBAL_SETTING
+            )
+            if (didPerform) return
+            // If not performed fall back to using the vibrator directly
+        }
+
+        val duration = prefs.inputFeedback.hapticVibrationDuration
+        if (duration != 0) {
+            val effectiveDuration = (duration * factor).toLong().coerceAtLeast(1L)
+            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
+                val strength = when {
+                    vibrator.hasAmplitudeControl() -> prefs.inputFeedback.hapticVibrationStrength
+                    else -> VibrationEffect.DEFAULT_AMPLITUDE
+                }
+                if (strength != 0) {
+                    val effectiveStrength = when {
+                        vibrator.hasAmplitudeControl() -> (255.0 * ((strength * factor) / 100.0)).toInt().coerceIn(1, 255)
+                        else -> strength
+                    }
+                    flogDebug { "Perform haptic with duration=$effectiveDuration and strength=$effectiveStrength" }
+                    val effect = VibrationEffect.createOneShot(effectiveDuration, effectiveStrength)
+                    vibrator.vibrate(effect)
+                }
+            } else {
+                flogDebug { "Perform haptic with duration=$effectiveDuration" }
+                @Suppress("DEPRECATION")
+                vibrator.vibrate(effectiveDuration)
+            }
+        }
+    }
+}

app/src/main/java/dev/patrickgold/florisboard/ime/keyboard/Key.kt

@@ -28,7 +28,7 @@ import android.graphics.Rect
  * @property data The base key data this key represents.This can be anything - from a basic text key to an emoji key
  *  to a complex selector.
  */
-abstract class Key(open val data: KeyData) {
+abstract class Key(open val data: AbstractKeyData) {
     /**
      * Specifies whether this key is enabled or not.
      */