Utils
๐ค AI-Generated Content
This documentation was generated with AI assistance and is still being audited. Some, or potentially a lot, of this information may be inaccurate. Learn more.
provide.foundation.utils
¶
Classes¶
ContextScopedCache
¶
Bases: Generic[K, V]
Thread-safe, async-safe cache scoped to context managers.
Unlike global LRU caches (for memoization), this provides isolated cache instances per execution context - ideal for recursive operations that need temporary storage without memory leaks.
The cache uses ContextVar for automatic thread/async isolation, and context managers for automatic cleanup. Nested contexts reuse the parent's cache to maintain consistency within an operation.
Examples:
>>> cache = ContextScopedCache[str, int]("user_ids")
>>>
>>> with cache.scope():
... cache.set("alice", 1)
... cache.set("bob", 2)
... print(cache.get("alice")) # 1
...
>>> # Cache is automatically cleared when exiting scope
>>> with cache.scope():
... print(cache.get("alice")) # None (fresh scope)
Nested contexts reuse parent cache:
>>> with cache.scope():
... cache.set("key", "outer")
... with cache.scope():
... print(cache.get("key")) # "outer" (same cache)
... cache.set("key", "inner")
... print(cache.get("key")) # "inner" (modified in nested scope)
Initialize a context-scoped cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Identifier for the cache (used in ContextVar name) |
'cache'
|
Source code in provide/foundation/utils/scoped_cache.py
Functions¶
clear
¶
Clear current context's cache.
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called outside a cache scope |
Source code in provide/foundation/utils/scoped_cache.py
contains
¶
Check if key exists in current context's cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
K
|
Cache key to check |
required |
Returns:
| Type | Description |
|---|---|
bool
|
True if key exists in cache |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called outside a cache scope |
Source code in provide/foundation/utils/scoped_cache.py
get
¶
Get value from current context's cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
K
|
Cache key |
required |
default
|
V | None
|
Value to return if key not found |
None
|
Returns:
| Type | Description |
|---|---|
V | None
|
Cached value or default |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called outside a cache scope |
Source code in provide/foundation/utils/scoped_cache.py
is_active
¶
Check if cache context is currently active.
Returns:
| Type | Description |
|---|---|
bool
|
True if inside a cache scope, False otherwise |
scope
¶
Create an isolated cache scope.
If a cache context already exists (nested call), reuses the existing cache. Otherwise, creates a new cache and cleans it up on exit.
Yields:
| Type | Description |
|---|---|
Generator[None]
|
None (use cache methods within the context) |
Source code in provide/foundation/utils/scoped_cache.py
set
¶
Set value in current context's cache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
K
|
Cache key |
required |
value
|
V
|
Value to cache |
required |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called outside a cache scope |
Source code in provide/foundation/utils/scoped_cache.py
size
¶
Get number of items in current context's cache.
Returns:
| Type | Description |
|---|---|
int
|
Number of cached items |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If called outside a cache scope |
Source code in provide/foundation/utils/scoped_cache.py
DependencyStatus
¶
Status of an optional dependency.
EnvPrefix
¶
Environment variable reader with prefix support.
Provides convenient access to environment variables with a common prefix, useful for application-specific configuration namespacing.
Uses caching to improve performance for repeated name lookups.
Examples:
>>> app_env = EnvPrefix('MYAPP')
>>> app_env.get_bool('DEBUG') # Reads MYAPP_DEBUG
>>> app_env['database_url'] # Reads MYAPP_DATABASE_URL
Initialize with prefix.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
prefix
|
str
|
Prefix for all environment variables |
required |
separator
|
str
|
Separator between prefix and variable name |
'_'
|
Source code in provide/foundation/utils/environment/prefix.py
Functions¶
__contains__
¶
__getitem__
¶
all_with_prefix
¶
Get all environment variables with this prefix.
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Dictionary of variable names (without prefix) to values |
Source code in provide/foundation/utils/environment/prefix.py
get_bool
¶
get_dict
¶
get_dict(
name: str,
default: dict[str, str] | None = None,
item_separator: str = ",",
key_value_separator: str = "=",
) -> dict[str, str]
Get dictionary with prefix.
Source code in provide/foundation/utils/environment/prefix.py
get_float
¶
get_int
¶
get_list
¶
get_path
¶
get_str
¶
require
¶
TokenBucketRateLimiter
¶
TokenBucketRateLimiter(
capacity: float,
refill_rate: float,
time_source: Callable[[], float] | None = None,
)
A Token Bucket rate limiter for asyncio applications.
This limiter allows for bursts up to a specified capacity and refills tokens at a constant rate. It is designed to be thread-safe using an asyncio.Lock.
Initialize the TokenBucketRateLimiter.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
capacity
|
float
|
The maximum number of tokens the bucket can hold (burst capacity). |
required |
refill_rate
|
float
|
The rate at which tokens are refilled per second. |
required |
time_source
|
Callable[[], float] | None
|
Optional callable that returns current time (for testing). Defaults to time.monotonic. |
None
|
Source code in provide/foundation/utils/rate_limiting.py
Functions¶
get_current_tokens
async
¶
Returns the current number of tokens, for testing/monitoring.
Source code in provide/foundation/utils/rate_limiting.py
is_allowed
async
¶
Check if a request is allowed based on available tokens.
This method is asynchronous and thread-safe. It refills tokens based on elapsed time and then attempts to consume a token.
Returns:
| Type | Description |
|---|---|
bool
|
True if the request is allowed, False otherwise. |
Source code in provide/foundation/utils/rate_limiting.py
Functions¶
__getattr__
¶
Lazy import for modules.
Source code in provide/foundation/utils/__init__.py
auto_parse
¶
Automatically parse value based on an attrs field's type and metadata.
This function first checks for a converter in the field's metadata, then falls back to type-based parsing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
attr
|
Any
|
attrs field (from fields(Class)) |
required |
value
|
str
|
String value to parse |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Parsed value based on field type or converter |
Examples:
>>> from attrs import define, field, fields
>>> @define
... class Config:
... count: int = field()
... enabled: bool = field()
... custom: str = field(converter=lambda x: x.upper())
>>> c = Config(count=0, enabled=False, custom="")
>>> auto_parse(fields(Config).count, "42")
42
>>> auto_parse(fields(Config).enabled, "true")
True
>>> auto_parse(fields(Config).custom, "hello")
'HELLO'
Source code in provide/foundation/parsers/attrs_integration.py
check_optional_deps
¶
check_optional_deps(
*, quiet: bool = False, return_status: bool = False
) -> list[DependencyStatus] | None
Check and display optional dependency status.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
quiet
|
bool
|
If True, don't print status (just return it) |
False
|
return_status
|
bool
|
If True, return the status list |
False
|
Returns:
| Type | Description |
|---|---|
list[DependencyStatus] | None
|
Optional list of dependency statuses if return_status=True |
Source code in provide/foundation/utils/deps.py
create_dependency_stub
¶
Create a stub class that raises DependencyError on instantiation or use.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
package
|
str
|
Name of the missing package (e.g., "httpx", "cryptography") |
required |
feature
|
str
|
Foundation feature name (e.g., "transport", "crypto") |
required |
Returns:
| Type | Description |
|---|---|
type
|
A stub class that raises DependencyError when instantiated or used |
Example
HTTPTransport = create_dependency_stub("httpx", "transport") transport = HTTPTransport() # Raises DependencyError with install instructions
Source code in provide/foundation/utils/stubs.py
create_function_stub
¶
Create a stub function that raises DependencyError when called.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
package
|
str
|
Name of the missing package (e.g., "httpx", "mkdocs") |
required |
feature
|
str
|
Foundation feature name (e.g., "transport", "docs") |
required |
Returns:
| Type | Description |
|---|---|
Any
|
A stub function that raises DependencyError when called |
Example
generate_docs = create_function_stub("mkdocs", "docs") generate_docs() # Raises DependencyError with install instructions
Source code in provide/foundation/utils/stubs.py
create_module_stub
¶
Create a stub module-like object that raises DependencyError on attribute access.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
package
|
str
|
Name of the missing package (e.g., "httpx") |
required |
feature
|
str
|
Foundation feature name (e.g., "transport") |
required |
Returns:
| Type | Description |
|---|---|
Any
|
A stub object that raises DependencyError on any attribute access |
Example
httpx = create_module_stub("httpx", "transport") httpx.AsyncClient() # Raises DependencyError with install instructions
Source code in provide/foundation/utils/stubs.py
get_available_features
¶
get_bool
¶
Get boolean environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
bool | None
|
Default value if not set |
None
|
Returns:
| Type | Description |
|---|---|
bool | None
|
Boolean value, None (if set but empty), or default (if unset) |
Note
Empty string is treated as ambiguous and returns None with a warning. Unset variable returns the default value.
Examples:
Source code in provide/foundation/utils/environment/getters.py
get_dict
¶
get_dict(
name: str,
default: dict[str, str] | None = None,
item_separator: str = ",",
key_value_separator: str = "=",
) -> dict[str, str]
Get dictionary from environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
dict[str, str] | None
|
Default dict if not set |
None
|
item_separator
|
str
|
Separator between items |
','
|
key_value_separator
|
str
|
Separator between key and value |
'='
|
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Dictionary of string key-value pairs |
Examples:
>>> os.environ['CONFIG'] = 'key1=val1,key2=val2'
>>> get_dict('CONFIG')
{'key1': 'val1', 'key2': 'val2'}
Source code in provide/foundation/utils/environment/getters.py
get_float
¶
Get float environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
float | None
|
Default value if not set |
None
|
Returns:
| Type | Description |
|---|---|
float | None
|
Float value or default |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If value cannot be parsed as float |
Source code in provide/foundation/utils/environment/getters.py
get_int
¶
Get integer environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
int | None
|
Default value if not set |
None
|
Returns:
| Type | Description |
|---|---|
int | None
|
Integer value or default |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If value cannot be parsed as integer |
Source code in provide/foundation/utils/environment/getters.py
get_list
¶
Get list from environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
list[str] | None
|
Default list if not set |
None
|
separator
|
str
|
String separator (default: comma) |
','
|
Returns:
| Type | Description |
|---|---|
list[str]
|
List of strings |
Examples:
Source code in provide/foundation/utils/environment/getters.py
get_optional_dependencies
¶
Get status of all optional dependencies.
Returns:
| Type | Description |
|---|---|
list[DependencyStatus]
|
List of dependency status objects |
Source code in provide/foundation/utils/deps.py
get_path
¶
Get path environment variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
default
|
Path | str | None
|
Default path if not set |
None
|
Returns:
| Type | Description |
|---|---|
Path | None
|
Path object or None |
Source code in provide/foundation/utils/environment/getters.py
get_str
¶
get_version
¶
Get the version for a package.
Reads from VERSION file if it exists, otherwise falls back to package metadata, then to default development version.
This function is thread-safe and caches results after the first call per package.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
package_name
|
str
|
The package name as it appears in PyPI (e.g., "provide-foundation") |
required |
caller_file
|
str | Path | None
|
Path to the calling module's file, used to find VERSION file. If None, uses the calling context. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The current version string |
Source code in provide/foundation/utils/versioning.py
has_dependency
¶
lazy_import
¶
Import a module lazily with comprehensive safety checks.
This function provides thread-safe lazy loading with protection against: - Circular imports (tracks import chains) - Stack overflow (enforces maximum depth) - Corrupted module states (validates sys.modules)
Commonly lazy-loaded modules: - cli: Requires optional 'click' dependency - crypto: Cryptographic utilities - formatting: Text formatting utilities - metrics: Metrics collection - observability: Observability features
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent_module
|
str
|
The parent module name (e.g., "provide.foundation") |
required |
name
|
str
|
Module name to lazy-load (e.g., "cli") |
required |
Returns:
| Type | Description |
|---|---|
object
|
The imported module |
Raises:
| Type | Description |
|---|---|
AttributeError
|
If module is not allowed for lazy loading or circular import detected |
ImportError
|
If module import fails |
RecursionError
|
If import depth exceeds safe limits |
Note
Complexity is intentionally high to handle all edge cases in this critical import hook (recursion, corruption, depth limits).
Example
from provide.foundation.utils.importer import lazy_import cli = lazy_import("provide.foundation", "cli")
Source code in provide/foundation/utils/importer.py
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | |
parse_bool
¶
Parse a boolean value from string or other types.
Accepts: true/false, yes/no, 1/0, on/off (case-insensitive)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Any
|
Value to parse as boolean |
required |
strict
|
bool
|
If True, only accept bool or string types (raise TypeError otherwise) |
False
|
Returns:
| Type | Description |
|---|---|
bool
|
Boolean value |
Raises:
| Type | Description |
|---|---|
TypeError
|
If strict=True and value is not bool or string, or if value is not bool/str |
ValueError
|
If value cannot be parsed as boolean |
Source code in provide/foundation/parsers/primitives.py
parse_dict
¶
parse_dict(
value: str | dict[str, str],
item_separator: str = ",",
key_separator: str = "=",
strip: bool = True,
) -> dict[str, str]
Parse a dictionary from a string.
Format: "key1=value1,key2=value2"
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str | dict[str, str]
|
String or dict to parse |
required |
item_separator
|
str
|
Separator between items |
','
|
key_separator
|
str
|
Separator between key and value |
'='
|
strip
|
bool
|
Whether to strip whitespace |
True
|
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Dictionary of string keys and values |
Raises:
| Type | Description |
|---|---|
ValueError
|
If format is invalid |
Source code in provide/foundation/parsers/collections.py
parse_duration
¶
Parse duration string to seconds.
Supports formats like: 30s, 5m, 2h, 1d, 1h30m, etc.
Results are cached for performance on repeated calls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str
|
Duration string |
required |
Returns:
| Type | Description |
|---|---|
int
|
Duration in seconds |
Examples:
Source code in provide/foundation/utils/environment/parsers.py
parse_list
¶
Parse a list from a string.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str | list[str]
|
String or list to parse |
required |
separator
|
str
|
Separator character |
','
|
strip
|
bool
|
Whether to strip whitespace from items |
True
|
Returns:
| Type | Description |
|---|---|
list[str]
|
List of strings |
Source code in provide/foundation/parsers/collections.py
parse_size
¶
Parse size string to bytes.
Supports formats like: 1024, 1KB, 10MB, 1.5GB, etc.
Results are cached for performance on repeated calls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str
|
Size string |
required |
Returns:
| Type | Description |
|---|---|
int
|
Size in bytes |
Examples:
Source code in provide/foundation/utils/environment/parsers.py
parse_typed_value
¶
Parse a string value to a specific type.
Handles basic types (int, float, bool, str) and generic types (list, dict). For attrs fields, pass field.type as target_type.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
str
|
String value to parse |
required |
target_type
|
type
|
Target type to convert to |
required |
Returns:
| Type | Description |
|---|---|
Any
|
Parsed value of the target type |
Examples:
>>> parse_typed_value("42", int)
42
>>> parse_typed_value("true", bool)
True
>>> parse_typed_value("a,b,c", list)
['a', 'b', 'c']
Source code in provide/foundation/parsers/typed.py
require
¶
Require an environment variable to be set.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Environment variable name |
required |
type_hint
|
type[T] | None
|
Optional type hint for parsing |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
Parsed value |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If variable is not set |
Source code in provide/foundation/utils/environment/getters.py
require_dependency
¶
Require a specific optional dependency, raise ImportError if missing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the dependency to require |
required |
Raises:
| Type | Description |
|---|---|
ImportError
|
If dependency is not available |
Source code in provide/foundation/utils/deps.py
reset_version_cache
¶
Reset the cached version for testing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
package_name
|
str | None
|
Specific package to reset, or None to reset all |
None
|
Warning
This should only be called from test code or test fixtures.
Source code in provide/foundation/utils/versioning.py
timed_block
¶
timed_block(
logger_instance: FoundationLogger,
event_name: str,
layer_keys: dict[str, Any] | None = None,
initial_kvs: dict[str, Any] | None = None,
**extra_kvs: Any,
) -> Generator[dict[str, Any], None, None]
Context manager that logs the duration of a code block.
Logs at DEBUG when entering, INFO on success, ERROR on exception.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
logger_instance
|
FoundationLogger
|
Logger to use for output |
required |
event_name
|
str
|
Name of the operation being timed |
required |
layer_keys
|
dict[str, Any] | None
|
Semantic layer keys (e.g., llm-specific keys) |
None
|
initial_kvs
|
dict[str, Any] | None
|
Initial key-value pairs to include in logs |
None
|
**extra_kvs
|
Any
|
Additional key-value pairs |
{}
|
Yields:
| Type | Description |
|---|---|
dict[str, Any]
|
A mutable dict that can be updated with additional context |
Example
with timed_block(logger, "database_query") as ctx: ctx["query"] = "SELECT * FROM users" result = db.query("SELECT * FROM users") ctx["rows"] = len(result)