[dale-wahl]

Adds Korean (and serendipitously some Chinese!) support to the text processors. Korean attaches particles and endings to words (학교, 학교에서 and 학교가 are apparently all "school"), which broke tokenizing and word matching.

Tokenise

• New Kiwi tokenizer for Korean (kiwipiepy, Apache-2.0). It splits words into morphemes, keeps the word classes you pick in dictionary form (nouns, verbs, adjectives, foreign words, and hashtags/mentions/emoji by default), and has its own Korean stop word list.
• New default tokenizer, "Recommended": Kiwi for Korean, jieba for Chinese, TweetTokenizer for everything else. Korean and Chinese are added to the language list.
• Asks for confirmation when the tokenizer doesn't fit the language, and refuses nltk word_tokenize for languages NLTK can't split into sentences (that used to crash).
• Sentence splitting for Korean (Kiwi) and Chinese, and Russian now actually uses razdel.
• Fixed along the way: Danish crashed on stop words, Portuguese and "Porter" got the wrong stop word lists, and jieba's spaces ended up as tokens.

Word matching (Filter by words or phrases, Over-time word counts)

• New "Match words" option: whole words (as before), start of words (for Korean) or anywhere (for Chinese). It's shared as TermMatcher in helpers, which also matches Korean Hangul stored as separate letters.
• Over-time word counts now finds terms with capitals. They never matched before.

Other

• Accent folding only removes accents from Latin, Greek, Arabic and Hebrew letters. Korean would have come out as separate letters, and Japanese, Hindi and Russian lost marks that change the word.
• Count values finds hashtags in any script.

Known limits: the Chinese sentence splitter can split inside links (see the note on urls_in_text), and Kiwi analyzes text twice when grouping by sentence. URLs in text is FREAKING HARD.

New dependency: kiwipiepy~=0.24.0, so existing installs need a pip install or an image rebuild. Tests are in tests/test_korean_text.py, and I checked it in the UI on the BEEP! Korean news comments dataset (hate speech dataset I found online).