One of my pandemic projects is to map the differences between a pointed and an unpointed text of the Hebrew Bible. I am doing this using the very memorable and capable left to right version of the square text called SimHebrew.
I had 'my own version' of a left-to right abbreviation in capital letters and with a few punctuation marks for the gutturals aleph and ayin. But I converted to the lower case version developed by Jonathan Orr-Stav. (Such a Latin-letter code is an abbreviation because it takes one byte as opposed to 7 bytes in rendered Unicode, and one byte as opposed to 2 internally. Using a Unicode database is really awkward for me and the technology was new 10 years ago so it was a non-starter.)
My method of data capture is to use the unpointed Mechon-Mamre text that can be downloaded from their site (one book at a time) and to run it through the SimHebrew converter here. Then to manipulate the text in notepad until I have legitimate insert statements for my database. This is somewhat prone to error but eventually I get a clean script. (I use a combination of Word and Notepad for the necessary global changes.)
After this I create a temporary table that matches: book, chapter, a conversion of the alef-betic verse number to a real verse number, and a word by word Hebrew word in SimHebrew, full pointed text word from the Leningrad codex, the stem code, raw word form, and semantic domain from my database, the word number relative to the start of the verse, and the word id (assigned by an Oracle sequence and connecting to my word table).
I recently added 2 Kings to my data. At first I assigned the wrong chapter numbers - a little oversight. That resulted in over 5000 differences in my calculations. When I fixed the issue, the differences went down to about 230. I then discovered that M-M is treating דִּבְיוֹנִ֖ים (dbivn) which they render as dung of a dove as two words (db ivnh - apparently a poor substitute for salt). Fixing those 4 instances dropped my mismatches by 100. Then word by word I work through the remainder until the differences are accounted for.
My program that calculates the words also suggests a code snippet that will fix any discrepancy (as long as I put it in the right place in the code.) The program as it stands gets 99% of the words right on first pass now. So my predicted SimHebrew Bible is about 99% right. Sorry - for the Bible, that's not good enough.
About this time in the history of the project, I ask, Is there a better way? I know I can finish the way I have begun and my brute force spelling changes are next to 0. I have over a third of the words in my test data now and over 75% of the stems represented. But the code is specific to prefix and suffix and sometimes to particular vowel combinations in the WLC. What are the real rules?
Native speakers who 'just know' the pronunciation - what are they really doing? Certainly they have retention in memory by word form, by context, and by stem. But can I get the program to discover the shortcuts that people use? And in some ways see what they are doing. (And thereby discover the nature of the evolution of language usage.)
All that is pretty straightforward. But what are the real rules?
Some of my rules are miles long conditions; strings of stems with prefix and suffix combinations and the occasional appeal to an odd Unicode value.
The 'rules' are long for when to render hireq as /i/.
I have been debating whether to design a data table for the rule combinations. I don't want to do it unless I cannot simplify the code. Last time I asked I discovered several simplifications. Looking for more...
Here's the rule for qamats becoming vav (RTL) or /o/ (SIM): I have converted the code to 'English'
Unconditionally for stems ahl, anih, arc, avn, bvw, ctl, grn, iq+n, iqwn, irqym, nvh, rnn, yziali, zvh,
(+ is my single letter internal abbreviation for tet ט.)
or the stem is +rk and prefix suffix combination in (none, cm)
or the stem is +hr and prefix suffix combination in (none, t, c, t)
or the stem is abd and prefix suffix combination in (b, n)
or the stem is acl and prefix suffix combination in (l, h, b, nu)
or the stem is adm and v_prefix = m and the first part of the word is m with a schwa
or the stem is amn and the word form is amn and there is no tsere under the second stem letter
or prefix suffix combination = b, h
or the stem is ark and v_suffix in (ti, vt)
or the stem is bit and v_suffix in (icm)
or the stem is bzz and prefix suffix combination in (i, vm)
or the stem is cvl and prefix suffix combination in (v, clu)
or the stem is csh and prefix suffix combination in (none, u)
or the stem is dbr and prefix suffix combination in (b, c)
or the stem is gal and prefix suffix combination in (l, c)
or the stem is gbh and prefix suffix combination = l, h
or the stem is gml and prefix suffix combination in (none, h,none, c)
or the stem is hrh and prefix suffix combination in (vh, tihm)
or the stem is ivn and prefix suffix combination in (none, none)
or the stem is iwr and prefix suffix combination in (none, o, b, o)
or the stem is imn and prefix suffix combination in (m, none)
or the stem is isd and prefix suffix combination in (b, i)
or the stem is knn and not (suffix in (u) and there's a qamats under the second letter) and the domain is not in (PERSON, LOCATION)
or the stem is kq and v_suffix not in (u)
or the stem is kgg and prefix suffix combination in (none, i)
or the stem is krb and prefix suffix combination in (none, h, l, h)
or the stem is krm and prefix suffix combination in (none, h, vb, h)
or the stem is kwc and prefix suffix combination in (none, i)
or the stem is kzq and prefix suffix combination in (b, h, b, nu)
or the stem is lcd and prefix suffix combination in (l, h)
or the stem is mlc and prefix suffix combination in (l, o, b, o, c, o)
or the stem is mvt and prefix suffix combination in (h, h)
or the stem is pyl and prefix suffix combination in (c, h)
or the stem is q+n and prefix suffix combination in (none, i)
or the stem is qdw and v_suffix in (im, iv)
or the stem is qra and prefix suffix combination in (none, nu, none, ic, none, im)
or the stem is qrk and prefix suffix combination is none, h
or the stem is rb and prefix suffix combination in (b, none)
or the stem is sll and prefix suffix combination in (none, vh)
or the stem is rbb and prefix suffix combination in (v, none)
or the stem is rbb and prefix suffix combination in (hb, none)
or the stem is rkb and prefix suffix combination not in (none, none)
or the stem is rkx and prefix suffix combination in (l, h)
or the stem is tmm and prefix suffix combination in (b, none) and the first two letters of the word are not m
or the stem is wby and prefix suffix combination in (m, h, l, h)
or the stem is wcb and prefix suffix combination in (b, o)
or the stem is wcr and prefix suffix combination is l, h
or the stem is wdd and prefix suffix combination in (i, m)
or the stem is wmr and prefix suffix combination in (a, none,vl, h,none, h)
or the stem is wp+ and prefix suffix combination in (none, h)
or the stem is wr and suffix in (c, rc)
or the stem is wrw and prefix suffix combination in (none, ih)
or the stem is xrp and prefix suffix combination in (none, h)
or the stem is ybd and prefix suffix combination in (l, h)
or the stem is yni and prefix suffix combination in (none, nu)
or the stem is yzz and prefix suffix combination in (none, i)
or the stem is zcr and prefix suffix combination in (b, nu)
or the stem is yzr and prefix suffix combination in (l, ni, l, u,none, ni,none, nu)
then
the qamats under the first letter of the stem becomes /o/.
And by the way if the stem is qdw and v_suffix in (im) then the schwa under the first letter becomes /o/!
(or maybe only v)