Wednesday, September 30, 2020

Unpointing the Hebrew text

I am still in the middle of an exercise to explore the differences between the Bible in unpointed text and the Westminster Leningrad Codex.

E.g. Judges 1:1

א,א ויהי, אחרי מות יהושוע, וישאלו בני ישראל, ביהוה לאמור:

or 

וַיְהִ֗י אַחֲרֵי֙ מ֣וֹת יְהוֹשֻׁ֔עַ וַֽיִּשְׁאֲלוּ֙ בְּנֵ֣י יִשְׂרָאֵ֔ל בַּיהוָ֖ה לֵאמֹ֑ר

And how would a computer transform the second to the first? Going the other way would be interesting too but impossible since information is destroyed in the mapping.

This exercise may take a full year. I recently removed an error in the testing environment. I was failing to see misplaced a's in the SimHebrew result. Interesting that the error affected only about 50 words in the first 140,000. My test data has doubled since the last report(s). As I add new books to the data, my routine gets about 99% of the transformation from the pointed WLC to unpointed Hebrew in the first pass.

The 140,000 words of the test data represent 3304 distinct stems, 82% of all the stems in the Biblical corpus.The data contains 25,000 of the roughly 40,000 distinct word forms used in the canon. There are 50 unresolvable words. These are words in the pointed text exactly identical to each other but which transform to differing forms. It seems to me that they are errors. But I may be wrong. There may be a contextual clue that my approach misses. I could transform them, but at present I am simply ignoring them. It may be that I just can't see the rule at this time. There are plenty of spelling idiosyncrasies in the WLC and I allow them to be respelled. 

The first few times I have attempted to explain the process have revealed to me how prone to complexity my thought process is. I want to get to a simpler explainable process. As long as it is this complex, I feel that I have not understood. The patterns bear little resemblance to the traditional paradigms that are taught in elementary Hebrew classes. When you have all the gadgets of a language to deal with at once, the amo, amas, amat approach does not seem to help much. The tension lies in the method of abstraction. 

If I work by vowel, I get a long string of traditional programming. This is what I am doing. What if I could work by stem? There are a few places where I do more than one operation related to a single stem, but there are too many stems (4087) to make the process explainable. Would an abstraction work by sets of stems as Lambdin does in his grammar? That's the question - could such an approach be developed and would it show the flower of the language?

I don't know yet.

What happens if we attempt to trace the rules followed by stem?

So at random, here is the first stem, ail. What an interesting choice (see the glosses below). This is a great word to start with. It is a word with multiple uses: for an animal (hart and ram), for strength, as a portion of a building, a portico, as the name of a location.

There are 3 rules for ail in my logic at the moment:

  1. i followed by qamats may turn into a double-i - so far this is only for words with prefix c and suffix vt or none, i.e. of the form c/!\vt, c/!\. Here ! stands for whatever is left of the stem in the word form, and /\ show the prefix and suffix.
  2. the first consonant a with patah followed by i will become double-i for words of the form /!\t, b/!\vt, c/!\, and c/!\vt. 
  3. tsere becomes i for several word forms /!\iv, /!\ihmh /!\, v/!\miv v/!\iv h/!\im.
  4. otherwise a word form for ail just transforms letter by letter.
Imagine designing a data abstraction for this - then could one find other stems that use the same rules? So far, the prospects for congruence or consolidation are slim.

If you have Hebrew in you, to write and to read, how would you manage these rules? 

Table of the transform of the stem ail from WLC into unpointed left to right SimHebrew

Ordered by WLC Word

Reference

word form

morph

sim

word

WLC word

Domain

translation

 

Psalms 88:5(8)

ail

ail

-aiil

־אֱיָל

STRENGTH

איל potency

Ezekiel 40:14(3)

ailim

ail\im

-ailim

־אֵילִים

CITY

אילים porticoes

2 Kings 16:6(8)

ailt

ail\t

-ailt

־אֵילַת

LOCATION

אילת Eilat

2 Kings 14:22(4)

ailt

ail\t

-ailt

־אֵילַת

LOCATION

אילת Eilat

Ezekiel 40:31(7)

ailiv

ail\iv

-ailiv

־אֵילָיו

CITY

איליו its portico

Ezekiel 40:37(6)

ailiv

ail\iv

-ailiv

־אֵילָיו

CITY

איליו its porticoes

Ezekiel 40:26(13)

ailiv

ail\iv

-ailiv

־אֵילָיו

CITY

איליו its porticoes

2 Chronicles 8:17(7)

ailvt

ail\vt

-ailvt

־אֵילוֹת

LOCATION

אילות Hart

2 Chronicles 26:2(4)

ailvt

ail\vt

-ailvt

־אֵילוֹת

LOCATION

אילות Hart

Ezekiel 40:14(7)

ail

ail

-ail

־אֵיל

CITY

איל the portico of

Ezekiel 40:34(6)

aliv

al\iv

-ailiv

־אֵלָיו

CITY

אליו its porticoes

Ezekiel 40:16(18)

ail

ail

-ail

־אַיִל

CITY

איל a portico

Psalms 22:1(3)

ailt

ail\t

-aiilt

־אַיֶּלֶת

ANIMAL

אילת the hart of

Ezekiel 40:49(15)

hailim

h/ail\im

-hailim

־הָאֵילִים

CITY

האילים the porticoes

Psalms 22:20(5)

ailvti

ail\vti

aiilvti

אֱיָלוּתִי

ANIMAL

אילותי my hart

Psalms 66:15(7)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

Micah 6:7(4)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

1 Chronicles 15:26(14)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

2 Kings 3:4(14)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

Job 42:8(7)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

1 Chronicles 29:21(12)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

2 Chronicles 29:32(9)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

2 Chronicles 17:11(13)

ailim

ail\im

ailim

אֵילִים

ANIMAL

אילים rams

2 Kings 24:15(14)

aili

ail\i

aili

אֵילֵי

STRENGTH

אילי the potentates of

2 Kings 16:6(16)

ailt

ail\t

ailt

אֵילַת

LOCATION

אילת Eilat

Ezekiel 40:24(9)

ailiv

ail\iv

ailiv

אֵילָיו

CITY

איליו portico

Ezekiel 40:16(6)

alihmh

al\ihmh

ailihmh

אֵלֵיהֵמָה

CITY

אליהמה their porticoes

Ezekiel 40:36(2)

aliv

al\iv

ailiv

אֵלָיו

CITY

אליו its porticoes

Ezekiel 40:48(6)

al

al

ail

אֵל

CITY

אל a portico of

Proverbs 5:19(1)

ailt

ail\t

aiilt

אַיֶּלֶת

ANIMAL

אילת the hart of

Job 39:1(7)

ailvt

ail\vt

aiilvt

אַיָּלוֹת

ANIMAL

אילות the hart

Psalms 29:9(4)

ailvt

ail\vt

aiilvt

אַיָּלוֹת

ANIMAL

אילות hart

Ezekiel 40:38(3)

bailim

b/ail\im

bailim

בְּאֵילִים

CITY

באילים were in the porticoes of

Song 3:5(7)

bailvt

b/ail\vt

baiilvt

בְּאַיְלוֹת

ANIMAL

באילות by the hart of

Song 2:7(7)

bailvt

b/ail\vt

baiilvt

בְּאַיְלוֹת

ANIMAL

באילות by the hart of

2 Chronicles 29:22(10)

halim

h/al\im

hailim

הָאֵלִים

ANIMAL

האלים the rams

1 Kings 6:31(8)

hail

h/ail

hail

הָאַיִל

CITY

האיל the portico

Song 2:17(13)

hailim

h/ail\im

haiilim

הָאַיָּלִים

ANIMAL

האילים of the hart

Song 8:14(8)

hailim

h/ail\im

haiilim

הָאַיָּלִים

ANIMAL

האילים the hart

Song 2:9(6)

hailim

h/ail\im

haiilim

הָאַיָּלִים

ANIMAL

האילים the hart

2 Chronicles 29:21(4)

vailim

v/ail\im

vailim

וְאֵילִים

ANIMAL

ואילים and rams

2 Chronicles 13:9(22)

vailm

v/ail\m

vailim

וְאֵילִם

ANIMAL

ואילם and rams

Ezekiel 40:22(14)

vailmv

v/ail\mv

vailmiv

וְאֵילַמָּיו

CITY

ואילמו and its portico

Ezekiel 40:9(7)

vailiv

v/ail\iv

vailiv

וְאֵילָיו

CITY

ואיליו and its porticoes

Ezekiel 40:29(2)

vailiv

v/ail\iv

vailiv

וְאֵילָיו

CITY

ואיליו and its porticoes

Ezekiel 40:21(6)

vailiv

v/ail\iv

vailiv

וְאֵילָיו

CITY

ואיליו and its porticoes

Ezekiel 40:37(1)

vailiv

v/ail\iv

vailiv

וְאֵילָיו

CITY

ואיליו and its porticoes

Ezekiel 40:22(2)

valmiv

v/al\miv

vailmiv

וְאֵלַמָּיו

CITY

ואלמיו and its porticoes

Ezekiel 40:33(2)

valiv

v/al\iv

vailiv

וְאֵלָיו

CITY

ואליו its porticoes

Psalms 114:6(3)

cailim

c/ail\im

cailim

כְאֵילִים

ANIMAL

כאילים like rams

Psalms 114:4(3)

cailim

c/ail\im

cailim

כְאֵילִים

ANIMAL

כאילים like rams

Jeremiah 51:40(4)

cailim

c/ail\im

cailim

כְּאֵילִים

ANIMAL

כאילים like rams

Lamentations 1:6(8)

cailim

c/ail\im

caiilim

כְּאַיָּלִים

ANIMAL

כאילים like hart

Psalms 42:2(1)

cail

c/ail

caiil

כְּאַיָּל

ANIMAL

כאיל as a hart

Habakkuk 3:19(6)

cailvt

c/ail\vt

caiilvt

כָּאַיָּלוֹת

ANIMAL

כאילות as the hart

Psalms 18:34(3)

cailvt

c/ail\vt

caiilvt

כָּאַיָּלוֹת

ANIMAL

כאילות the hart

Ezekiel 40:10(14)

lailm

l/ail\m

lailim

לָאֵילִם

CITY

לאילם for the porticoes

2 Kings 16:6(13)

mailvt

m/ail\vt

mailvt

מֵאֵילוֹת

LOCATION

מאילות from Eilat

1 Kings 5:3(10)

mail

m/ail

maiil

מֵאַיָּל

ANIMAL

מאיל hart

Curious that in both Hebrew and English in the list above, there are words where one cannot distinguish the singular from the plural.