Unpointing the Hebrew text

I am still in the middle of an exercise to explore the differences between the Bible in unpointed text and the Westminster Leningrad Codex.

E.g. Judges 1:1

א,א ויהי, אחרי מות יהושוע, וישאלו בני ישראל, ביהוה לאמור:

or 

וַיְהִ֗י אַחֲרֵי֙ מ֣וֹת יְהוֹשֻׁ֔עַ וַֽיִּשְׁאֲלוּ֙ בְּנֵ֣י יִשְׂרָאֵ֔ל בַּיהוָ֖ה לֵאמֹ֑ר

And how would a computer transform the second to the first? Going the other way would be interesting too but impossible since information is destroyed in the mapping.

This exercise may take a full year. I recently removed an error in the testing environment. I was failing to see misplaced a's in the SimHebrew result. Interesting that the error affected only about 50 words in the first 140,000. My test data has doubled since the last report(s). As I add new books to the data, my routine gets about 99% of the transformation from the pointed WLC to unpointed Hebrew in the first pass.

The 140,000 words of the test data represent 3304 distinct stems, 82% of all the stems in the Biblical corpus.The data contains 25,000 of the roughly 40,000 distinct word forms used in the canon. There are 50 unresolvable words. These are words in the pointed text exactly identical to each other but which transform to differing forms. It seems to me that they are errors. But I may be wrong. There may be a contextual clue that my approach misses. I could transform them, but at present I am simply ignoring them. It may be that I just can't see the rule at this time. There are plenty of spelling idiosyncrasies in the WLC and I allow them to be respelled. 

The first few times I have attempted to explain the process have revealed to me how prone to complexity my thought process is. I want to get to a simpler explainable process. As long as it is this complex, I feel that I have not understood. The patterns bear little resemblance to the traditional paradigms that are taught in elementary Hebrew classes. When you have all the gadgets of a language to deal with at once, the amo, amas, amat approach does not seem to help much. The tension lies in the method of abstraction. 

If I work by vowel, I get a long string of traditional programming. This is what I am doing. What if I could work by stem? There are a few places where I do more than one operation related to a single stem, but there are too many stems (4087) to make the process explainable. Would an abstraction work by sets of stems as Lambdin does in his grammar? That's the question - could such an approach be developed and would it show the flower of the language?

I don't know yet.

What happens if we attempt to trace the rules followed by stem?

So at random, here is the first stem, ail. What an interesting choice (see the glosses below). This is a great word to start with. It is a word with multiple uses: for an animal (hart and ram), for strength, as a portion of a building, a portico, as the name of a location.

There are 3 rules for ail in my logic at the moment:

1. i followed by qamats may turn into a double-i - so far this is only for words with prefix c and suffix vt or none, i.e. of the form c/!\vt, c/!\. Here ! stands for whatever is left of the stem in the word form, and /\ show the prefix and suffix.
2. the first consonant a with patah followed by i will become double-i for words of the form /!\t, b/!\vt, c/!\, and c/!\vt. 
3. tsere becomes i for several word forms /!\iv, /!\ihmh /!\, v/!\miv v/!\iv h/!\im.
4. otherwise a word form for ail just transforms letter by letter.

Imagine designing a data abstraction for this - then could one find other stems that use the same rules? So far, the prospects for congruence or consolidation are slim.

If you have Hebrew in you, to write and to read, how would you manage these rules? 

Table of the transform of the stem ail from WLC into unpointed left to right SimHebrew Ordered by WLC Word
Reference	word form	morph	sim word	WLC word	Domain	translation
Psalms 88:5(8)	ail	ail	-aiil	־אֱיָל	STRENGTH	איל potency
Ezekiel 40:14(3)	ailim	ail\im	-ailim	־אֵילִים	CITY	אילים porticoes
2 Kings 16:6(8)	ailt	ail\t	-ailt	־אֵילַת	LOCATION	אילת Eilat
2 Kings 14:22(4)	ailt	ail\t	-ailt	־אֵילַת	LOCATION	אילת Eilat
Ezekiel 40:31(7)	ailiv	ail\iv	-ailiv	־אֵילָיו	CITY	איליו its portico
Ezekiel 40:37(6)	ailiv	ail\iv	-ailiv	־אֵילָיו	CITY	איליו its porticoes
Ezekiel 40:26(13)	ailiv	ail\iv	-ailiv	־אֵילָיו	CITY	איליו its porticoes
2 Chronicles 8:17(7)	ailvt	ail\vt	-ailvt	־אֵילוֹת	LOCATION	אילות Hart
2 Chronicles 26:2(4)	ailvt	ail\vt	-ailvt	־אֵילוֹת	LOCATION	אילות Hart
Ezekiel 40:14(7)	ail	ail	-ail	־אֵיל	CITY	איל the portico of
Ezekiel 40:34(6)	aliv	al\iv	-ailiv	־אֵלָיו	CITY	אליו its porticoes
Ezekiel 40:16(18)	ail	ail	-ail	־אַיִל	CITY	איל a portico
Psalms 22:1(3)	ailt	ail\t	-aiilt	־אַיֶּלֶת	ANIMAL	אילת the hart of
Ezekiel 40:49(15)	hailim	h/ail\im	-hailim	־הָאֵילִים	CITY	האילים the porticoes
Psalms 22:20(5)	ailvti	ail\vti	aiilvti	אֱיָלוּתִי	ANIMAL	אילותי my hart
Psalms 66:15(7)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
Micah 6:7(4)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
1 Chronicles 15:26(14)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
2 Kings 3:4(14)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
Job 42:8(7)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
1 Chronicles 29:21(12)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
2 Chronicles 29:32(9)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
2 Chronicles 17:11(13)	ailim	ail\im	ailim	אֵילִים	ANIMAL	אילים rams
2 Kings 24:15(14)	aili	ail\i	aili	אֵילֵי	STRENGTH	אילי the potentates of
2 Kings 16:6(16)	ailt	ail\t	ailt	אֵילַת	LOCATION	אילת Eilat
Ezekiel 40:24(9)	ailiv	ail\iv	ailiv	אֵילָיו	CITY	איליו portico
Ezekiel 40:16(6)	alihmh	al\ihmh	ailihmh	אֵלֵיהֵמָה	CITY	אליהמה their porticoes
Ezekiel 40:36(2)	aliv	al\iv	ailiv	אֵלָיו	CITY	אליו its porticoes
Ezekiel 40:48(6)	al	al	ail	אֵל	CITY	אל a portico of
Proverbs 5:19(1)	ailt	ail\t	aiilt	אַיֶּלֶת	ANIMAL	אילת the hart of
Job 39:1(7)	ailvt	ail\vt	aiilvt	אַיָּלוֹת	ANIMAL	אילות the hart
Psalms 29:9(4)	ailvt	ail\vt	aiilvt	אַיָּלוֹת	ANIMAL	אילות hart
Ezekiel 40:38(3)	bailim	b/ail\im	bailim	בְּאֵילִים	CITY	באילים were in the porticoes of
Song 3:5(7)	bailvt	b/ail\vt	baiilvt	בְּאַיְלוֹת	ANIMAL	באילות by the hart of
Song 2:7(7)	bailvt	b/ail\vt	baiilvt	בְּאַיְלוֹת	ANIMAL	באילות by the hart of
2 Chronicles 29:22(10)	halim	h/al\im	hailim	הָאֵלִים	ANIMAL	האלים the rams
1 Kings 6:31(8)	hail	h/ail	hail	הָאַיִל	CITY	האיל the portico
Song 2:17(13)	hailim	h/ail\im	haiilim	הָאַיָּלִים	ANIMAL	האילים of the hart
Song 8:14(8)	hailim	h/ail\im	haiilim	הָאַיָּלִים	ANIMAL	האילים the hart
Song 2:9(6)	hailim	h/ail\im	haiilim	הָאַיָּלִים	ANIMAL	האילים the hart
2 Chronicles 29:21(4)	vailim	v/ail\im	vailim	וְאֵילִים	ANIMAL	ואילים and rams
2 Chronicles 13:9(22)	vailm	v/ail\m	vailim	וְאֵילִם	ANIMAL	ואילם and rams
Ezekiel 40:22(14)	vailmv	v/ail\mv	vailmiv	וְאֵילַמָּיו	CITY	ואילמו and its portico
Ezekiel 40:9(7)	vailiv	v/ail\iv	vailiv	וְאֵילָיו	CITY	ואיליו and its porticoes
Ezekiel 40:29(2)	vailiv	v/ail\iv	vailiv	וְאֵילָיו	CITY	ואיליו and its porticoes
Ezekiel 40:21(6)	vailiv	v/ail\iv	vailiv	וְאֵילָיו	CITY	ואיליו and its porticoes
Ezekiel 40:37(1)	vailiv	v/ail\iv	vailiv	וְאֵילָיו	CITY	ואיליו and its porticoes
Ezekiel 40:22(2)	valmiv	v/al\miv	vailmiv	וְאֵלַמָּיו	CITY	ואלמיו and its porticoes
Ezekiel 40:33(2)	valiv	v/al\iv	vailiv	וְאֵלָיו	CITY	ואליו its porticoes
Psalms 114:6(3)	cailim	c/ail\im	cailim	כְאֵילִים	ANIMAL	כאילים like rams
Psalms 114:4(3)	cailim	c/ail\im	cailim	כְאֵילִים	ANIMAL	כאילים like rams
Jeremiah 51:40(4)	cailim	c/ail\im	cailim	כְּאֵילִים	ANIMAL	כאילים like rams
Lamentations 1:6(8)	cailim	c/ail\im	caiilim	כְּאַיָּלִים	ANIMAL	כאילים like hart
Psalms 42:2(1)	cail	c/ail	caiil	כְּאַיָּל	ANIMAL	כאיל as a hart
Habakkuk 3:19(6)	cailvt	c/ail\vt	caiilvt	כָּאַיָּלוֹת	ANIMAL	כאילות as the hart
Psalms 18:34(3)	cailvt	c/ail\vt	caiilvt	כָּאַיָּלוֹת	ANIMAL	כאילות the hart
Ezekiel 40:10(14)	lailm	l/ail\m	lailim	לָאֵילִם	CITY	לאילם for the porticoes
2 Kings 16:6(13)	mailvt	m/ail\vt	mailvt	מֵאֵילוֹת	LOCATION	מאילות from Eilat
1 Kings 5:3(10)	mail	m/ail	maiil	מֵאַיָּל	ANIMAL	מאיל hart

Curious that in both Hebrew and English in the list above, there are words where one cannot distinguish the singular from the plural.