Singleton

big_corporation_inc.id_translation.singleton

Convenience functions using the Translator singleton.

Functions

extract_names(translatable, *[, ...])	Extract names in translatable.
fetch([translatable, names, ignore_names, ...])	Fetch translations.
get_singleton(*[, recreate])	Get the Translator singleton instance.
go_offline([translatable, names, ...])	Retrieve and cache translations in memory.
initialize_sources([task_id, force])	Perform source discovery (fetcher initialization).
map(translatable[, names, ignore_names, ...])	Map names to translation sources.
map_scores(translatable[, names, ...])	Returns raw match scores for name-to-source mapping.
translate(translatable[, names, ...])	Translate IDs to human-readable strings.
translated_names([with_source])	Return the names translated by the last successful translate()-call.

translate(translatable, names=None, *, ignore_names=None, copy=True, override_function=None, max_fails=1.0, reverse=False, fmt=None, io_kwargs=None)

Translate IDs to human-readable strings.

Note

Convenience method. Calls id_translation.Translator.translate() using the current get_singleton() instance. See below for original docstring.

Simplified process:

1. The extract_names() method derives names of the translatable to translate (if needed).

2. The map() method performs name-to-source mapping (see DirectionalMapping).

3. The fetch() method extracts IDs to translate and retrieves data (see TranslationMap).

4. Finally, the translate() method (i.e. this one) applies the translations and returns to the caller.

See the Translation primer page for a detailed process description.

Parameters:

• translatable – A data structure to translate.

• names – Explicit names to translate. Derive from translatable if None. Alternatively, you may pass a dict on the form {name_in_translatable: source_to_use}.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• copy – If False, translate in-place and return None.

• override_function – A callable (name, sources, context) -> Source | None, where context is always None. See Mapper.apply for details.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled. Missing values (NaN, null) in pandas, dask, polars and pyarrow data are pass-throughs and never count; in a builtin collection or array, None and NaN are ordinary (unknown) IDs, and do.

• reverse – If True, perform translations back to IDs. Offline mode only.

• fmt – A format string such as ‘{id}:{name}’ to use. Default is Translator.fmt.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

Returns:

A translated copy of translatable if copy=True, otherwise None.

Examples

Manual name-to-source mapping with a temporary name-only Format.

>>> n2s = {"lions": "animals", "big_cats": "animals"}
>>> translator.translate({"lions": 2, "big_cats": 2}, names=n2s, fmt="{name}")
{'lions': 'Simba', 'big_cats': 'Simba'}

Name mappings must be complete; any name not present in the keys will be ignored (left as-is).

Raises:

• UntranslatableTypeError – If type(translatable) cannot be translated.

• MissingNamesError – If names are not given and cannot be derived from translatable.

• UnmappedExplicitNamesError – If any required (explicitly given) names fail to map to a source.

• MappingError – If name-to-source mapping fails, e.g. ScoringDisabledError for a name that is neither a source nor overridden, when using the default Mapper.

• ValueError – If max_fails is not a valid fraction, or if reverse=True and several IDs of a source share a translation.

• TooManyFailedTranslationsError – If translation fails for more than max_fails of IDs.

• ConnectionStatusError – If reverse=True while the Translator is online.

• UserMappingError – If override_function returns a source which is not known, and mapper.on_unknown_user_override='raise'.

See also

🔑 This is a key event method. See Key Event Records for details.

The ID_TRANSLATION_DISABLED variable.

translated_names(with_source=True)

Return the names translated by the last successful translate()-call.

Note

Convenience method. Calls id_translation.Translator.translated_names() using the current get_singleton() instance. See below for original docstring.

A call is successful if it translates at least one name. Calls that raise or abort leave the result unchanged: when no name maps to a source, or when ID_TRANSLATION_DISABLED is set.

Parameters:

with_source – If True, return a dict {name: source} instead of a list.

Returns:

Recent names translated by this Translator, in arbitrary order.

Raises:

ValueError – If no names have been translated using this Translator.

Notes

🧵 This method is not thread safe. See Thread safety for details.

extract_names(translatable, *, ignore_names=None, io_kwargs=None, raising=False)

Extract names in translatable.

Note

Convenience method. Calls id_translation.Translator.extract_names() using the current get_singleton() instance. See below for original docstring.

Perform name extraction. The names are the first input to the map() method, with the second being the sources. Does not call initialize_sources().

Parameters:

• translatable – A data structure to map names for.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

• raising – If False (the default), return None instead of raising MissingNamesError when name extraction fails.

Returns:

Names to map to sources.

Raises:

MissingNamesError – If names cannot be derived (only when raising=True).

map(translatable, names=None, *, ignore_names=None, override_function=None, io_kwargs=None)

Map names to translation sources.

Note

Convenience method. Calls id_translation.Translator.map() using the current get_singleton() instance. See below for original docstring.

Parameters:

• translatable – A data structure to map names for.

• names – Explicit names to translate. Derive from translatable if None. Alternatively, you may pass a dict on the form {name_in_translatable: source_to_use}.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, context) -> Source | None, where context is always None. See Mapper.apply for details.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO).

Returns:

A mapping of names to translation sources.

Raises:

• MissingNamesError – If names are not given and cannot be derived from translatable.

• UnmappedExplicitNamesError – If any required (explicitly given) names fail to map to a source.

• MappingError – If name-to-source mapping fails, e.g. ScoringDisabledError for a name that is neither a source nor overridden, when using the default Mapper.

• UserMappingError – If override_function returns a source which is not known, and mapper.on_unknown_user_override='raise'.

See also

🔑 This is a key event method. See Key Event Records for details.

The extract_names() method.

map_scores(translatable, names=None, *, ignore_names=None, override_function=None, io_kwargs=None)

Returns raw match scores for name-to-source mapping. See map() for details.

Note

Convenience method. Calls id_translation.Translator.map_scores() using the current get_singleton() instance. See below for original docstring.

fetch(translatable=None, names=None, *, ignore_names=None, override_function=None, max_fails=1.0, fmt=None, io_kwargs=None)

Fetch translations.

Note

Convenience method. Calls id_translation.Translator.fetch() using the current get_singleton() instance. See below for original docstring.

Calling fetch without arguments will perform a Fetcher.fetch_all -operation, without going offline.

The returned TranslationMap may be converted to native types with TranslationMap.to_dicts.

Parameters:

• translatable – A data structure to translate.

• names – Explicit names to translate. Derive from translatable if None. Alternatively, you may pass a dict on the form {name_in_translatable: source_to_use}.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, context) -> Source | None, where context is always None. See Mapper.apply for details. Ignored when translatable is given.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled. Missing values (NaN, null) in pandas, dask, polars and pyarrow data are pass-throughs and never count; in a builtin collection or array, None and NaN are ordinary (unknown) IDs, and do.

• fmt – A format string such as ‘{id}:{name}’ to use. Default is Translator.fmt.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO). Ignored, with a warning, when translatable is None; this will raise in id-translation==2.0.0.

Returns:

A TranslationMap.

Raises:

ConnectionStatusError – If disconnected from the fetcher, i.e. not online.

Examples

Using the returned TranslationMap class.

>>> translation_map = translator.fetch()
>>> translation_map
TranslationMap('animals': 3 IDs, 'people': 2 IDs)

Convert to finished translations.

• TranslationMap.to_translations → {source: MagicDict}, where a MagicDict is similar to a regular dict[IdType, str]-type dict.

>>> people = translation_map.to_translations()["people"]
>>> people
{1999: '1999:Sofia', 1991: '1991:Richard'}

Warning

The MagicDict class is used internally and has a few important differences from the built-in type. Please refer to the MagicDict class documentation for details.

To convert a MagicDict to a regular dict, simply use the dict constructor:

>>> dict(people)
{1999: '1999:Sofia', 1991: '1991:Richard'}

Convert to raw translation data.

• TranslationMap.to_pandas → {source: DataFrame}

• TranslationMap.to_dicts → {source: {placeholder: [values...]}}

>>> translation_map.to_dicts()["people"]
{'id': [1999, 1991], 'name': ['Sofia', 'Richard']}

Deprecated since version 1.3.0: Passing io_kwargs without translatable; ignored with a FutureWarning, will raise in id-translation==2.0.0.

initialize_sources(task_id=None, *, force=None)

Perform source discovery (fetcher initialization).

Note

Convenience method. Calls id_translation.Translator.initialize_sources() using the current get_singleton() instance. See below for original docstring.

This method does nothing if the Translator isn’t online.

Warning

Fetcher-provided transformers are queried by the first call only (ignores force).

Parameters:

• task_id – Used for logging.

• force –

  If True, perform full discovery even if sources are already known.

  Deprecated since version 1.4.0: Removed in 2.0.0. Build a new Translator instead.

Returns:

Self, for chained assignment.

See also

🧵 Call this before sharing a Translator between threads; see Thread safety.

go_offline(translatable=None, names=None, *, ignore_names=None, override_function=None, max_fails=1.0, fmt=None, io_kwargs=None, path=None)

Retrieve and cache translations in memory.

Note

Convenience method. Calls id_translation.Translator.go_offline() using the current get_singleton() instance. See below for original docstring.

Warning

The fetcher will be destroyed.

Parameters:

• translatable – Data from which IDs to fetch will be extracted. Fetch all IDs if None.

• names – Explicit names to translate. Derive from translatable if None.

• ignore_names – Names not to translate, or a predicate (NameType) -> bool.

• override_function – A callable (name, sources, context) -> Source | None, where context is always None. See Mapper.apply for details. Ignored when translatable is given.

• max_fails – The maximum fraction of IDs for which translation may fail. 1=disabled. Missing values (NaN, null) in pandas, dask, polars and pyarrow data are pass-throughs and never count; in a builtin collection or array, None and NaN are ordinary (unknown) IDs, and do.

• fmt – A format string such as ‘{id}:{name}’ to use. Default is Translator.fmt. The snapshot is only guaranteed to hold the placeholders of fmt when translatable is given, and those of Translator.fmt otherwise; list any others that a later translate() needs there as optional blocks, e.g. '{id}:{name}[ {team}]'.

• io_kwargs – Keyword arguments for the IO class (e.g. PandasIO). Ignored, with a warning, when translatable is None; this will raise in id-translation==2.0.0.

• path – If given, serialize the Translator to disk after retrieving data.

Returns:

Self, for chained assignment.

Raises:

• ForbiddenOperationError – If Fetcher.fetch_all is disabled and translatable=None.

• MappingError – If map() fails (only when translatable is given).

Notes

🧵 This method is not thread safe. See Thread safety for details.

The Translator is guaranteed to be serializable() once offline. Fetchers often aren’t as they require things like database connections to function.

See also

🔑 This is a key event method. See Key Event Records for details.

The restore() method (when path is set).

Deprecated since version 1.3.0: Calling this method when already offline (returns immediately), and passing io_kwargs without translatable (ignored). Both emit FutureWarning and will raise in id-translation==2.0.0.

get_singleton(*, recreate=False)

Get the Translator singleton instance.

The exact type returned depends on TRANSLATOR_TYPE. By default, this is the regular id_translation.Translator type provided by id_translation.

Thread safe: the instance is created, and its sources initialized, under a lock before it is shared. Creation includes source discovery, so call this once during startup rather than from the first request.

Parameters:

recreate – If True, force recreating the current singleton instance.

Returns:

A id_translation.Translator.

Raises:

TimeoutError – If creation by another thread outlasts SINGLETON_LOCK_TIMEOUT.