Singleton

big_corporation_inc.id_translation.singleton

Convenience functions using the Translator singleton.

Functions

extract_names(translatable, *[, ...])	Extract names in translatable.
fetch([translatable, names, ignore_names, ...])	Fetch translations.
get_singleton(*[, recreate])	Get the Translator singleton instance.
go_offline([translatable, names, ...])	Retrieve and cache translations in memory.
initialize_sources([task_id, force])	Perform source discovery (fetcher initialization).
map(translatable[, names, ignore_names, ...])	Map names to translation sources.
map_scores(translatable[, names, ...])	Returns raw match scores for name-to-source mapping.
translate(translatable[, names, ...])	Translate IDs to human-readable strings.
translated_names([with_source])	Return the names that were translated by the most recent translate()-call.

translate(translatable, names=None, *, ignore_names=None, copy=True, override_function=None, max_fails=1.0, reverse=False, fmt=None, io_kwargs=None)

Translate IDs to human-readable strings.

Note

Convenience method. Calls id_translation.Translator.translate() using the current get_singleton() instance. See below for original docstring.

Simplified process:

1. The extract_names() method derives names of the translatable to translate (if needed).

2. The map() method performs name-to-source mapping (see DirectionalMapping).

3. The fetch() method extracts IDs to translate and retrieves data (see TranslationMap).

4. Finally, the translate() method (i.e. this one) applies the translations and returns to the caller.

See the Translation primer page for a detailed process description.

See also

🔑 This is a key event method. See Key Event Records for details.

Parameters:

• translatable – A data structure to translate.

• names – Explicit names to translate. Derive from translatable if None. Alternatively, you may pass a dict on the form {name_in_translatable: source_to_use}.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• copy – If False, translate in-place and return None.

• override_function – A callable (name, sources, ids) -> Source | None. See Mapper.apply for details.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled. Missing IDs (e.g. NaN) in pandas types are pass-throughs and never count; a None in a builtin collection is an ordinary – unknown – ID, and does.

• reverse – If True, perform translations back to IDs. Offline mode only.

• fmt – A format string such as ‘{id}:{name}’ use. Default is Translator.fmt.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

Returns:

A translated copy of translatable if copy=True, otherwise None.

Examples

Manual name-to-source mapping with a temporary name-only Format.

>>> n2s = {"lions": "animals", "big_cats": "animals"}
>>> translator.translate({"lions": 2, "big_cats": 2}, names=n2s, fmt="{name}")
{'lions': 'Simba', 'big_cats': 'Simba'}

Name mappings must be complete; any name not present in the keys will be ignored (left as-is).

Raises:

• UntranslatableTypeError – If type(translatable) cannot be translated.

• MissingNamesError – If names are not given and cannot be derived from translatable.

• UnmappedExplicitNamesError – If any required (explicitly given) names fail to map to a source.

• MappingError – If name-to-source mapping is ambiguous.

• ValueError – If max_fails is not a valid fraction.

• TooManyFailedTranslationsError – If translation fails for more than max_fails of IDs.

• ConnectionStatusError – If reverse=True while the Translator is online.

• UserMappingError – If override_function returns a source which is not known, and mapper.on_unknown_user_override != 'ignore'.

See also

The ID_TRANSLATION_DISABLED variable.

translated_names(with_source=True)

Return the names that were translated by the most recent translate()-call.

Note

Convenience method. Calls id_translation.Translator.translated_names() using the current get_singleton() instance. See below for original docstring.

Parameters:

with_source – If True, return a dict {name: source} instead of a list.

Returns:

Recent names translated by this Translator, in arbitrary order.

Raises:

ValueError – If no names have been translated using this Translator.

Notes

🧵 This method is not thread safe. See Thread safety for details.

extract_names(translatable, *, ignore_names=None, io_kwargs=None, raising=False)

Extract names in translatable.

Note

Convenience method. Calls id_translation.Translator.extract_names() using the current get_singleton() instance. See below for original docstring.

Perform name extraction. The names are the first input to the map() method, with the second being the sources. Does not call initialize_sources().

Parameters:

• translatable – A data structure to map names for.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

• raising – If False (the default), return None instead of raising MissingNamesError when name extraction fails.

Returns:

Names to map to sources.

Raises:

MissingNamesError – If names cannot be derived (only when raising=True).

map(translatable, names=None, *, ignore_names=None, override_function=None, io_kwargs=None)

Map names to translation sources.

Note

Convenience method. Calls id_translation.Translator.map() using the current get_singleton() instance. See below for original docstring.

Parameters:

• translatable – A data structure to map names for.

• names – Explicit names to translate. Derive from translatable if None.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, ids) -> Source | None. See Mapper.apply for details.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

Returns:

A mapping of names to translation sources. Returns None if mapping failed.

Raises:

• MissingNamesError – If names are not given and cannot be derived from translatable.

• UnmappedExplicitNamesError – If any required (explicitly given) names fail to map to a source.

• MappingError – If name-to-source mapping is ambiguous.

• UserMappingError – If override_function returns a source which is not known, and mapper.on_unknown_user_override != 'ignore'.

See also

🔑 This is a key event method. See Key Event Records for details.

See also

The extract_names() method.

map_scores(translatable, names=None, *, ignore_names=None, override_function=None, io_kwargs=None)

Returns raw match scores for name-to-source mapping. See map() for details.

Note

Convenience method. Calls id_translation.Translator.map_scores() using the current get_singleton() instance. See below for original docstring.

fetch(translatable=None, names=None, *, ignore_names=None, override_function=None, max_fails=1.0, fmt=None, io_kwargs=None)

Fetch translations.

Note

Convenience method. Calls id_translation.Translator.fetch() using the current get_singleton() instance. See below for original docstring.

Calling fetch without arguments will perform a Fetcher.fetch_all -operation, without going offline.

The returned TranslationMap may be converted to native types with TranslationMap.to_dicts.

Parameters:

• translatable – A data structure to translate.

• names – Explicit names to translate. Derive from translatable if None. Alternatively, you may pass a dict on the form {name_in_translatable: source_to_use}.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, ids) -> Source | None. See Mapper.apply for details.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled.

• fmt – A format string such as ‘{id}:{name}’ use. Default is Translator.fmt.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO). Ignored, with a warning, when translatable is None; this will raise in id-translation==2.0.0.

Returns:

A TranslationMap.

Raises:

ConnectionStatusError – If disconnected from the fetcher, i.e. not online.

Examples

Using the returned TranslationMap class.

>>> translation_map = translator.fetch()
>>> translation_map
TranslationMap('animals': 3 IDs, 'people': 2 IDs)

Convert to finished translations.

• TranslationMap.to_translations → {source: MagicDict}, where a MagicDict is similar to a regular dict[IdType, str]-type dict.

>>> people = translation_map.to_translations()["people"]
>>> people
{1999: '1999:Sofia', 1991: '1991:Richard'}

Warning

The MagicDict class is used internally and has a few important differences from the built-in type. Please refer to the MagicDict class documentation for details.

To convert to a MagicDict to a regular dict, simply use the dict constructor:

>>> dict(people)
{1999: '1999:Sofia', 1991: '1991:Richard'}

Convert to raw translation data.

• TranslationMap.to_pandas → {source: DataFrame}

• TranslationMap.to_dicts → {source: {placeholder: [values...]}}

>>> translation_map.to_dicts()["people"]
{'id': [1999, 1991], 'name': ['Sofia', 'Richard']}

Deprecated since version 1.3.0: Passing io_kwargs without translatable; ignored with a FutureWarning, will raise in id-translation==2.0.0.

initialize_sources(task_id=None, *, force=False)

Perform source discovery (fetcher initialization).

Note

Convenience method. Calls id_translation.Translator.initialize_sources() using the current get_singleton() instance. See below for original docstring.

This method does nothing if the Translator isn’t online.

Warning

Fetcher-provided transformers are queried by the first call only (ignores force).

Parameters:

• task_id – Used for logging.

• force – If True, perform full discovery even if sources are already known.

Returns:

Self, for chained assignment.

See also

🧵 Call this before sharing a Translator between threads; see Thread safety.

go_offline(translatable=None, names=None, *, ignore_names=None, override_function=None, max_fails=1.0, fmt=None, io_kwargs=None, path=None)

Retrieve and cache translations in memory.

Note

Convenience method. Calls id_translation.Translator.go_offline() using the current get_singleton() instance. See below for original docstring.

Warning

The fetcher will be destroyed.

Parameters:

• translatable – Data from which IDs to fetch will be extracted. Fetch all IDs if None.

• names – Explicit names to translate. Derive from translatable if None.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, ids) -> Source | None. See Mapper.apply for details.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled.

• fmt – A format string such as ‘{id}:{name}’ use. Default is Translator.fmt.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO). Ignored, with a warning, when translatable is None; this will raise in id-translation==2.0.0.

• path – If given, serialize the Translator to disk after retrieving data.

Returns:

Self, for chained assignment.

Raises:

• ForbiddenOperationError – If Fetcher.fetch_all is disabled and translatable=None.

• MappingError – If map() fails (only when translatable is given).

Notes

🧵 This method is not thread safe. See Thread safety for details.

The Translator is guaranteed to be serializable() once offline. Fetchers often aren’t as they require things like database connections to function.

See also

🔑 This is a key event method. See Key Event Records for details.

The restore() method (when path is set).

Deprecated since version 1.3.0: Calling this method when already offline (returns immediately), and passing io_kwargs without translatable (ignored). Both emit FutureWarning and will raise in id-translation==2.0.0.

get_singleton(*, recreate=False)

Get the Translator singleton instance.

The exact type returned depends on TRANSLATOR_TYPE. By default, this is the regular id_translation.Translator type provided by id_translation.

Parameters:

recreate – If True, force recreating the current singleton instance.

Returns:

A id_translation.Translator.

Notes

Creating the instance is not thread safe, and neither is the first initialize_sources() call it performs – that call resolves the sources and registers fetcher-provided transformers, mutating the Translator. Since translate() triggers it implicitly, the first concurrent translate() call through the singleton is the hazard. Warm the singleton once during startup:

get_singleton().initialize_sources()

before handing it to worker threads. See https://id-translation.readthedocs.io/en/stable/documentation/translation-concurrency.html#thread-safety.