OpenITI NgramReaderDHFAIS App{endix} 1
No search yet.

How to use

  1. Type a search line: a label, then the word forms, e.g. #Tasalṭana #tslTn. Arabic letters work too: #Tasalṭana #تسلطن.
  2. Check the forms found. The app lists every matching form with its count; untick any you don't mean.
  3. Compare words by adding more lines; each line becomes one curve.
  4. Explore the tabs: over time, book types, centuries, books, authors. Every chart explains itself in its caption.
  5. Share or cite: “Copy link” reproduces your exact search; “Cite these results” gives a ready-made citation.

About and instructions

Like the Google Ngram Viewer, the OpenITI NgramReader charts the frequencies of words and phrases over time, using the OpenITI corpus (8,468 texts; 1,054,622,781 tokens). Unlike it, the NgramReader lets you combine different forms of the same word into one curve — kitāb, al-kitāb, wa-l-kitāb — and build classes of terms, such as Baġdād together with Madīnat al-salām, or all cities of Ḫurāsān. By default, searches run on about 95% of the corpus: 414 texts (61 mln tokens) still contain paraeditorial elements and are left out unless you choose keep all texts (100%). Time is given in periods of 25 years by the author's death date (AH).

Search lines

Each line is one curve: #Label #item1 #item2 …. The first # part is the label shown in the legend; the other parts are the items searched for, and their matches are added together. For example, #Baġdād #bgdAd #bbgdAd #wbgdAd #wbbgdAd, or shorter with a regular expression, #Baġdād #[wb]?bgdAd. Add as many lines as you like with + add line.

Book types

The bars show how the hits of each line divide over the book types: Together in one chart, Separate with a panel per line, always with the types in alphabetical order, here and in every other chart. Compared with corpus divides a line's share of hits in a type by the type's share of the tokens of the selected corpus, on a log scale. The table uses the three-letter type codes; hold the pointer on a code for its name; a darker cell is a larger value within its line. Classic (radar) keeps the earlier chart.

How to read Compared with corpus

Some kinds of books, such as biographical dictionaries, make up a large part of the corpus, so most words appear in them often simply because there is so much of them. This chart removes that effect. For each book type it compares the share of your search's hits found there with that type's share of the whole corpus.

Example (made-up numbers): if chronicles hold 8 % of all words in the corpus but 16 % of the hits for your search, the bar for chronicles reaches 2×.

Bars based on only a few hits can be misleading; hover over a bar, or tap it, to see how many hits it rests on. Light bars rest on fewer than 20 hits.

Typing labels

While you type the label of a line (the first # part), a letter with special forms brings up a small row of them: t offers t, ṭ, ṯ, ŧ; a ā, á, ã; d ḍ, ḏ; g ġ; h ḥ, ḫ; i ī; s ṣ, š; u ū; z ẓ; the apostrophe ' ʿ, ʾ. Tap one, or press Tab to step through them, and keep typing. Items, after the second #, are not changed.

Transliteration and Arabic input

Items are matched against the corpus in simplified Buckwalter transliteration: ء = c, ا إ أ آ = A, ب = b, ة = o, ت = t, ث = v, ج = j, ح = H, خ = x, د = d, ذ = V, ر = r, ز = z, س = s, ش = E, ص = S, ض = D, ط = T, ظ = Z, ع = C, غ = g, ف = f, ق = q, ك = k, ل = l, م = m, ن = n, ه = h, ؤ ئ = c, و = w, ى ي = y. You can also type items in Arabic script: letters are converted automatically, short vowels (harakat) and tatweel are removed, and a space between Arabic words becomes _. Labels stay as you type them.

Unigrams, bigrams, trigrams

Words of a phrase are joined with _: kataba ilay-hi is ktb_Alyh. The number of _ sets the length: none = unigram, one = bigram, two = trigram. All items of a search must have the same length.

Choosing forms

After Search, every line lists the forms its items match, with their Arabic script and corpus count. Untick forms you do not want and press Update results; only ticked forms are counted. The page address keeps the whole search, ticks included, so you can bookmark or share it.

Regular expressions

Items are regular expressions in JavaScript syntax, matched against whole words: each line becomes \b(item1|item2)\b. The usual constructs work as in the earlier R version of this app: [wb]?, (Al)?, \w+, ., *, +, {2,3}. Some ICU constructs of the R version are not available: possessive quantifiers (a++), atomic groups ((?>…)), \Q…\E quoting, POSIX classes such as [[:alpha:]], and inline flags such as (?i); the app says so when a search uses them. Matching is case-sensitive (s = س, S = ص). Spaces inside items are ignored.

Book titles and author names

The books and authors tables show a readable title and author name, with the Arabic, beside the OpenITI URI. Book titles and author names are working labels and may contain errors; the URI is authoritative. The labels were generated with a language model; a raised ? marks a label flagged as uncertain. Without a label the URI is shown. A book title or author name opens its profile in the OpenITI Book Classifier (book typology, rolling assessment, similar books) in a new tab; the text link opens the text in the OpenITI release. Each results table has a filter box that ignores case and diacritics.

Data download

The first search for unigrams, bigrams or trigrams downloads that list of forms once (3 MB for unigrams, about 27 MB for bigrams or trigrams); later searches reuse it. Counts are fetched only for the forms you search.

Cite these results