From one question to many literatures

A language model (Gemini, when configured) turns your question into short keyword queries in 18 languages (English, Chinese, Spanish, Portuguese, French, German, Japanese, Russian, Korean, Indonesian, Italian, Turkish, Polish, Ukrainian, Arabic, Hindi, Afrikaans and Swahili), using the terminology papers in each language tend to use. The first fifteen are the largest abstract-bearing literatures in OpenAlex; Hindi, Afrikaans and Swahili are the largest from South Asia and Africa, smaller (about 2,800 to 15,000 works with abstracts each) but real. Your original question is always kept and searched too.

Translations are machine-generated, and technical meaning can shift between languages. Every query variant is shown in the Search coverage panel. Without an LLM, only the original query is searched, and the interface says so.

Each variant becomes one OpenAlex title-and-abstract search that retrieves up to 100 candidates: as many as the results list holds, so any single language could fill the whole list on its own. No language is capped below that. A translated query is restricted to papers whose OpenAlex language tag matches it, so the Chinese query looks for Chinese-language papers. Year and publication-language filters are sent to the API. If a query returns fewer than five papers, it is broadened step by step to its two most essential terms, then to the single core term. Ranking then reorders the broader set by relevance.

Responses are cached for 24 hours. A paper’s language starts as OpenAlex’s tag, which OpenAlex detects automatically and sometimes gets wrong. A local detector flags papers whose abstract (or title) disagrees with their tag, unless the title itself is in the tagged language (many non-English journals add an English abstract), and Gemini reads those texts in one batch. Only when Gemini confirms a different language is the tag overridden. The badge is then dashed, and its tooltip names OpenAlex’s original tag. Language is never inferred from author names, venues or affiliations, and it never affects ranking.

Results are merged by normalized DOI, then OpenAlex ID, then an exact normalized title match with the same publication year. Records with different DOIs are never merged. Each paper keeps a list of every query variant that retrieved it and at what rank. That list drives the “via Chinese query” labels and the coverage statistics.

Candidates are then drawn from each language in proportion to how much literature it has on the topic. Gemini names the question’s most distinctive subject term in every language (asthma, 哮喘, asma, …), and one OpenAlex request counts the works in each language that match any of them. If that request fails, each language’s overall share of OpenAlex is used. This is relevant-document distribution estimation, as used in federated search. Each language keeps at least its two best matches. Equal numbers per language would oversample small literatures: for “Shakespeare”, English has about 99,000 matches and Italian about 1,100, and equal shares would let Italian fill as much of the list as English whenever relevance can’t tell papers apart. All kept candidates are then scored together and the top 100 (or 25 or 50) are shown, so the mix reflects where the research is. “Load more” fetches the next page in every language and re-ranks the whole set, up to 1,000 results; earlier pages, translations and scores come from the cache, so each step only scores the new papers.

Semantic: your question (embedded in every language it was translated into, then averaged) and each paper’s title and abstract are embedded with paraphrase-multilingual-MiniLM-L12-v2, a sentence-transformer trained to place paraphrases in 50+ languages near each other. It runs locally on the server. Score = cosine similarity. Text is truncated to 128 tokens, the model’s training length, as in its reference configuration. When there are more than three times as many candidates as results, a fast title-only pass first shortlists the top three times as many under any ranking mode, without regard to language; the shortlist is then scored on title and abstract. Each paper is scored on the better of its title alone and its title plus abstract, because abstracts add methods and numbers that dilute similarity. Scoring only title plus abstract would rank papers without an abstract above comparable papers that have one.

Keyword: BM25 (k1 = 1.2, b = 0.75) over titles and abstracts. Text is segmented with ICU word boundaries, which handle Chinese and Japanese words rather than single characters. Each paper is scored against every query variant and keeps its best score, so a Spanish paper is matched against the Spanish query. Each score is divided by that query’s maximum possible score, giving the share of an ideal match on the same 0–1 scale in every language. Without this, the rarity of each language’s words in a mostly-English pool would inflate every non-English score. Length is compared only among papers of the same kind (title-only or title plus abstract), and scoring always uses all of a language’s concepts, even when its search was broadened.

Hybrid: 0.7 × semantic + 0.3 × keyword, after min-max normalizing both within the result set. These weights are tunable starting values, not validated constants. Publication language, venue, citation counts and author information are never used in scoring.

Explanations are generated only from computed facts. “Strong semantic match” means cosine ≥ 0.6, and “moderate” means ≥ 0.4. Those thresholds are heuristics for this model. The other explanations are the matched query terms, the query variants that retrieved the paper, and whether only the title was available.

Titles and abstracts can be machine-translated into your query language. Translations are always labeled and shown alongside the original, never in place of it. Optional AI summaries use only the abstract and say “Not stated in the abstract” rather than guessing.

Comparison mode ranks the original-language query’s results and the full multilingual pool with the same scores, then shows their top-10 overlap and the papers that only the multilingual search found. Additional papers are not necessarily relevant.