Materials
For this study, a new dataset was compiled by aggregating several existing datasets on South-East Asian languages spoken in Southern China. The core of the dataset is a collection of 25 Hmong-Mien language varieties documented by Chén (2012). This dataset was standardized in an earlier study (Wu et al., 2020) by converting it to the standard formats recommended by the Cross-Lingustic Data Formats initiative (CLDF, Forkel et al., 2018). Using the CLDFBench toolkit (Forkel & List, 2020) allows regular and transparent data conversion to CLDF including links to reference catalogs, such as Glottolog for language varieties (Hammarström et al., 2021) and Concepticon for concepts (List et al., 2021a). In addition, CLDF makes transcriptions transparent by linking segments to the B(road)IPA transcription system, which is a stricter version of the standard International Phonetic Alphabet (IPA, 1999), for the representation of speech sounds (Anderson et al., 2018; List et al., 2021b).
Having shown earlier that CLDF greatly facilitates the aggregation of data from diverse sources (List et al., 2018), we assembled data on additional South-East Asian language varieties from sources which were either already converted to CLDF in earlier works (Běijīng Dàxué, 1964) or prepared specifically for this study (Wang, 2004). Table 1 shows the eight core datasets that were used in this study. In addition, Chinese dialect data is available in the form of lists of character pronunciations. While these do not provide any strict information on actual words, since individual characters correspond to morphemes in Chinese, it can still be useful to include them in larger multilingual collections, since they may fill gaps where the available data on words in Chinese dialects is sparse. For this reason, the character readings from two datasets (Běijīng Dàxué, 1962; Hóu, 2004) were roughly linked to common concepts in our base datasets in order to increase the coverage for individual language varieties.
Table 1. Sources of the data selected for this study.
Note that due to the rather low coverage of data without base list of 250 concepts in some datasets, several varieties were aggregated from two or more sources. This is indicated in the CLDF version of the dataset. For character readings, where rudimentary Concepticon mapping was carried out on a very selective basis, only the number of concepts linked to Concepticon is shown in the Source column, also indicated by an asterisk.
| ID | Source | Family | Varieties | Concepts |
|---|
| Source | Selected | Source | Selected |
|---|
| beidasinitic | Běijīng Dàxué (1964) | Chinese dialects | 18 | 6 | 905 | 146 |
| beidazihui | Běijīng Dàxué (1962) | Chinese dialects (characters) | 19 | 4 | *518 | 171 |
| castrosui | Castro & Pan (2015) | Sui dialects (Tai-Kadai) | 16 | 3 | 608 | 211 |
| castroyi | Castro et al. (2010) | Loloish dialects (Sino-Tibetan) | 6 | 1 | 540 | 222 |
| castrozhuang | Castro & Hansen (2010) | Zhuang dialects (Tai-Kadai) | 20 | 8 | 511 | 243 |
| chenhmongmien | Chén (2012) | Hmong-Mien | 25 | 23 | 888 | 250 |
| housinitic | Hóu (2004) | Chinese dialects | 40 | 10 | 180 | 61 |
| houzihui | Hóu (2004) | Chinese dialects (characters) | 40 | 9 | *155 | 77 |
| liusinitic | Liú et al. (2007) | Chinese dialects | 19 | 5 | 201 | 130 |
| wangbai | Wang (2004) | Bai dialects (Sino-Tibetan) | 9 | 1 | 471 | 144 |
From these datasets, a subset of 48 language varieties (23 Hmong-Mien languages, 11 Tai-Kadai languages, and 14 Sino-Tibetan languages) and 250 concepts were selected. The criterion for data and concept selection was the general coverage of individual language varieties with respect to the 250 concepts chosen, and the geographic proximity of the languages in the sample. While the coverage with respect to the concepts for which there is a word form in individual language varieties is reasonably high in most cases, there are some outliers, mostly from Chinese dialects, in which more than 40% of the concepts are missing. While a low coverage for certain varieties would be problematic for phylogenetic studies (Sagart et al., 2019), it was nevertheless decided to keep these language varieties in the sample, mostly because their geographic position makes them interesting candidates for donor and recipient languages in the current sample. Figure 1 shows the geographic distribution of the languages in our sample.
Figure 1. Languages in the sample.
While it would be desirable to extend the data further, we consider the current collection as sufficient for initial experiments on automated borrowing detection. Extending the dialect data, specifically for the Chinese dialects, could in theory be achieved in the future by integrating information on individual dialects extracted from dialect dictionaries, such as the collection of Lǐ (2002), which comprises dialect dictionaries for 42 dialect varieties. The integration of this and similar sources, however, cannot be done in a straightforward way, since the data was usually not collected from established questionnaires, but is instead listed only by dialect headwords (given by Chinese characters with their pronunciations), which are furthermore only sporadically translated (see Kurpaska, 2010: 128–183 on the structure of the dictionary collection by Lǐ, 2002). As a result, finding the counterpart for a concept like “sun” requires to search actively for a word that is potentially cognate with “sun” in the target dialect in a first step, and to verify this entry in a second step with the translation provided in the resource. Thus, although it would be desirable to improve on our current data basis, this endeavour would largely exceed the scope of the current study.
Methods
Automated borrowing detection. The new method for the identification of borrowings in multilingual wordlists is based on a very straightforward workflow that proceeds in two steps. In the first step, traditional methods for cognate detection are used to identify language-family-internal cognate sets in the data. In the second step, all language-internal cognate sets are compared across language families and clustered into sets of potentially borrowed words once the overall average distance among the cognate sets is below a certain threshold. For the first step, it is useful to use a conservative cognate detection method which searches for deep genealogical similarities among word forms. An example for such a method is the LexStat algorithm for automated cognate detection, which has been proposed earlier (see List, 2012a and List et al., 2017), or its modification, which searches for partial cognates (words which are not entirely cognate but only in parts of their individual morphemes) instead of full cognates (List et al., 2016). For the second step, it is useful to employ a less conservative method for cognate detection that searches for superficial phonetic similarities rather than deep similarities based on regular sound correspondences. Here, the sound-class-based alignment (SCA) method for pairwise and multiple phonetic alignment (List, 2012b) is a good candidate, specifically also because studies on pairwise borrowing detection have shown that SCA outperforms edit distance in this task (Zhang et al., 2021).
For our specific use case, we decided to use the LexStat algorithm adjusted for partial cognate detection (List et al., 2016), since it is well known that South-East Asian languages show frequent compounding patterns which cannot be captured when searching for full cognates in the data (for a detailed discussion on partial cognacy, we refer interested readers to Hill & List, 2017). The LexStat Partial algorithm expects words to be segmented into morphemes by the user and then uses a network approach to cluster morphemes which ocurr in the same concept slot of a given wordlist into sets of cognate morphemes, corresponding to partially cognate words. Since one would expect, however, that borrowings involve full words, it seems useful to employ a full word alignment algorithm for the second stage. When searching for partial cognates in a first instance, this means that one needs to find a way to convert partial cognates to full cognates later. Since words can share cognate morphemes in different parts, the partial cognate relation is not transitive (word A with form xyz can be partially cognate to word B with form x and to word C with form ya, while word B is not partially cognate with word C, but a word D with form ab would be cognate with D), a specific conversion procedure that transitivizes cognate relations is needed. For this purpose, we decided to use a new method that we recently developed (Wu & List, 2021). This method is based on a greedy algorithm that assigns partial cognates to full cognate sets which have at least one cognate set in common. This method starts from the most frequently recurring partial cognate, assigning all words which contain this morpheme to the same cognate sets, and then proceeds with the remaining word forms which have not yet been assigned to a cognate set until all words have been visited (yielding the merger of A with B due to their shared cognate set x and then proceed to cluster C and D due to their shared cognate a).
In order to compare two cognate sets expressing the same meaning from two different language families, the new method proposed here first computes pairwise SCA distances for each possible word pair assembled from languages from different language families. The distances are all stored in memory and then averaged. If the average distance is lower than a user-defined threshold, a link between the cognate sets is drawn. After all cognate sets from different languages have been compared in this fashion, the method searches for all connected components in this cognate set network and assigns all cognate sets appearing in the same connected component to the same set of potentially borrowed words.
This method does not resolve the direction of borrowings. But instead of earlier approaches, which only identify pairs of potentially borrowed words, it allows to cluster words into xenologs, that is, sets of words which are not entirely related by common descent, but also by lateral transfer (List, 2016). Furthermore, since all our methods needs as evidence is to find that two words are attested in different language families (following the idea first expressed in van der Ark et al., 2007), the method might even detect borrowings from a third language family, with the original word (or cognate forms thereof) not being reflected in the sample.
Figure 2 shows a rudimentary workflow example in which the major steps are displayed. In step (1), partial cognates are inferred with the help of the LexStat-Partial algorithm, for each language family in separation. In step (2) the greedy algorithm by Wu & List (2021) is used to convert partial to full cognates based on the identification of those cognate morphemes which recur in the largest part of the data. In step (3), individual cognate sets across language families are compared with each other using the SCA algorithm for pairwise alignments. Based on these scores (not shown in the figure), a network of cognate sets is constructed in which cognate sets are connected which show an average distance score beyond a user-defined threshold.
Figure 2. Workflow example.
The selection of thresholds is important for the identification of partial cognates and for the identification of cross-language-family cognate sets (or sets of xenologous words). Since both thresholds are crucially intertwined with each other, it is not easy to come up with an objective solution for threshold detection. Increasing the threshold for cognate detection, for example, will cluster more words into the same cognate set in related languages. The larger the language-internal cognate sets, however, the higher the chance that borrowing sare rejected if the second threshold is considerably low. Since threshold detection in cognate detection tasks is a problem that has no straightforward solution by now, all we can do for now is to derive thresholds by means of trial and error. While this may not seem optimal, we emphasize that we consider the major value of our approach not in the detection of indisputible truths regarding cognate and borrowed words, but rather as a tool supposed to aid linguists in their research, which would necessarily invole a manual refinement of automatic analyses.
An alternative way to achieve results similar to our workflow would be to use the well-established cognate detection algorithms on the whole dataset without splitting the data into language families first, and then infer potential borrowings by identifying those cognate sets which appear in more than one language family. While this approach has been used successfully in a computer-assisted study in the past (Hantgan et al., 2020), our approach has several advantages, as it allows us not only to combine different methods for language-family-internal and language-family-external cognate detection, but also runs faster, since a much smaller part of the data has to be compared in a one-to-one fashion. Since the implementation of this alternative method for borrowing detection can be done in a straightforward way with the help of the LingPy library, we added the code to the code package accompanying this study.
Annotation of borrowings in multilingual wordlists. In order to allow us to test the new method against human judgments, the data was annotated manually, using the EDICTOR (List, 2017), a web-based tool for curation of etymological data in historical linguistics. The data was annotated in two stages. First, cognate sets were identified inside all language families in our sample. In a second stage, cognate sets were themselves assembled into larger sets of potentially borrowed words.
An example for this annotation is shown in Figure 3, where words for "face" in Hmong-Mien and Sino-Tibetan are compared with each other. The table shows our annotation procedure and how data are displayed in the EDICTOR application The ID is used to refer to the original data point and allows us to trace the data from the original sources and across different files. The DOCULECT column provides a language identifier, which also provides rudimentary subgroup information (also displayed in the column SUBGROUP), which was made available with the most recent version of EDICTOR. The column TOKENS shows the sound sequences in segmented, normalized form, segmenting the individual sounds (which may be transcribed by more than one characters) by a space and uses the symbol + as an additional character to indicate morpheme boundaries. Partial cognate identifiers are manually added to the data in the column COGIDS, which are provided on a language-family-internal basis only and indicate with the help of unique identifiers which morphemes of a word form a group of xenologous words. The column MORPHEMES indicates which parts of the word express the main (or salient aspect of the) meaning "FACE" and help us to convert the partial cognates into full cognates, following Wu & List (2021) in taking the salient (or main) morphemes as the criterion to convert partial to full cognates. Column UBORID provides the user judgment on potentially borrowed words. Where known, the SOURCE of a borrowing was also indicated, as illustrated for the FACE by 面, Middle Chinese mjienH (Baxter, 1992). Additionally, information on the language family, to which a language belongs, is displayed in the column FAMILY, which is routinely provided in the CLDF datasets, from which the data was aggregated.
Figure 3. EDICTOR annotation of borrowings.
Data Formats and Data Representation. Having been collected from datasets originally provided in the unifying CLDF formats, which standardizes elicitation glosses for concepts via Concepticon (List et al., 2021a), language names via Glottolog (Hammarström et al., 2021), and phonetic transcriptions via CLTS (List et al., 2021b), the aggregated dataset along with our manual and automated analyses is also provided in the form of a CLDF dataset, which is curated on GitHub (https://github.com/lexibank/seabor) and archived with Zenodo (Version 1.0, https://doi.org/10.5281/zenodo.5037101). For a detailed introduction of the tabular CLDF formats, we refer the readers to Forkel et al. (2018).
As mentioned in the previous section, the annotation of partial cognates, full cognates, and borrowing candidates is carried out with the help of the EDICTOR tool. The format underlying this tool is based on a rather well-established tabular representation of lexical data in which each row of a table is reserved for one word form in a particular language and columns provide individual data on each word form. The EDICTOR database is curated online and can be directly accessed in read-only form by interested scholars (https://digling.org/links/seabor.html). For more information on the format, we refer interested readers to an initial study by Hill & List (2017), where partial cognacy and morpheme glosses were first discussed in detail, as well as the follow-up studies by Schweikhard & List (2020), where these formats were further extended, and Wu & List (2021), where a detailed discussion of the problem of partial cognacy is given.
Implementation. The new method was implemented as part of the LingRex Python package (List & Forkel, 2021a, Version 1.2). LingRex is an extension of LingPy which offers code that is specifically useful for the detection of sound correspondence patterns (List, 2019b) and the prediction of words which have not been elicited from cognate sets (Bodt & List, 2021). With this study, LingRex was further extended by the new method for the identification of borrowed words. The methods used for the efficient manual annotation of borrowed words were introduced as part of the most recent version (2.0.0) of the EDICTOR tool (List, 2021). Plots made in this study were carried out with the help of CLDFViz (Forkel, 2021), a Python package which facilitates the static and interactive visualization of data provided in CLDF, which was expanded to allow for the specific requirements for this study. The supplementary material accompanying this study offers both the data and the code needed to replicate the experiments which are reported in this study. Users who wish to replicate the study reported here can find all necessary information in the file “workflow.md”, where all steps of the workflow (including the creation of figures) are explained.
