Skip to content

Settings

Configuration management

DataJoint configuration system using pydantic-settings.

This module provides strongly-typed configuration with automatic loading from environment variables, secrets directories, and JSON config files.

Configuration sources (in priority order):

  1. Environment variables (DJ_*)
  2. Secrets directories (.secrets/ in project, /run/secrets/datajoint/)
  3. Project config file (datajoint.json, searched recursively up to .git/.hg)

Examples:

>>> import datajoint as dj
>>> dj.config.database.host
'localhost'
>>> dj.config.database.backend
'mysql'
>>> dj.config.database.port  # Auto-detects: 3306 for MySQL, 5432 for PostgreSQL
3306
>>> with dj.config.override(safemode=False):
...     # dangerous operations here
...     pass

Project structure::

myproject/
├── .git/
├── datajoint.json      # Project config (commit this)
├── .secrets/           # Local secrets (gitignore this)
│   ├── database.password
│   └── aws.secret_access_key
└── src/
    └── analysis.py     # Config found via parent search

find_config_file 

find_config_file(start=None)

Search for datajoint.json in current and parent directories.

Searches upward from start until finding the config file or hitting a project boundary (.git, .hg) or filesystem root.

Parameters:

Name Type Description Default
start Path

Directory to start search from. Defaults to current working directory.

 None

Returns:

Type Description
Path or None

Path to config file if found, None otherwise.

find_secrets_dir 

find_secrets_dir(config_path=None)

Find the secrets directory.

Priority:

  1. .secrets/ in same directory as datajoint.json (project secrets)
  2. /run/secrets/datajoint/ (Docker/Kubernetes secrets)

Parameters:

Name Type Description Default
config_path Path

Path to datajoint.json if found.

 None

Returns:

Type Description
Path or None

Path to secrets directory if found, None otherwise.

read_secret_file 

read_secret_file(secrets_dir, name)

Read a secret value from a file in the secrets directory.

Parameters:

Name Type Description Default
secrets_dir Path or None

Path to secrets directory.

 required
name str

Name of the secret file (e.g., 'database.password').

 required

Returns:

Type Description
str or None

Secret value as string, or None if not found.

DatabaseSettings

Bases: BaseSettings

Database connection settings.

set_default_port_from_backend 

set_default_port_from_backend()

Set default port based on backend if not explicitly provided.

ConnectionSettings

Bases: BaseSettings

Connection behavior settings.

DisplaySettings

Bases: BaseSettings

Display and preview settings.

StoresSettings

Bases: BaseSettings

Unified object storage configuration.

Stores configuration supports both hash-addressed and schema-addressed storage using the same named stores with _hash and _schema sections.

JobsSettings

Bases: BaseSettings

Job queue configuration for AutoPopulate 2.0.

Config

Bases: BaseSettings

Main DataJoint configuration.

Settings are loaded from (in priority order):

  1. Environment variables (DJ_*)
  2. Secrets directory (.secrets/ or /run/secrets/datajoint/)
  3. Config file (datajoint.json, searched in parent directories)
  4. Default values

Examples:

Access settings via attributes:

>>> config.database.host
>>> config.safemode

Override temporarily with context manager:

>>> with config.override(safemode=False):
...     pass

set_logger_level classmethod 

set_logger_level(v)

Update logger level when loglevel changes.

convert_path classmethod 

convert_path(v)

Convert string paths to Path objects.

get_store_spec 

get_store_spec(store=None, *, use_filepath_default=False)

Get configuration for a storage store.

Parameters:

Name Type Description Default
store str

Name of the store to retrieve. If None, uses the appropriate default.

 None
use_filepath_default bool

If True and store is None, uses stores.filepath_default instead of stores.default. Use for filepath references which are not part of OAS. Default: False (use stores.default for integrated storage).

 False

Returns:

Type Description
dict[str, Any]

Store configuration dict with validated fields.

Raises:

Type Description
DataJointError

If store is not configured or has invalid config.

load 

load(filename)

Load settings from a JSON file.

Parameters:

Name Type Description Default
filename str or Path

Path to load configuration from.

 required

override 

override(**kwargs)

Temporarily override configuration values.

Parameters:

Name Type Description Default
**kwargs Any

Settings to override. Use double underscore for nested settings (e.g., database__host="localhost").

 {}

Yields:

Type Description
Config

The config instance with overridden values.

Examples:

>>> with config.override(safemode=False, database__host="test"):
...     # config.safemode is False here
...     pass
>>> # config.safemode is restored

save_template staticmethod 

save_template(path='datajoint.json', minimal=True, create_secrets_dir=True)

Create a template datajoint.json configuration file.

Credentials should NOT be stored in datajoint.json. Instead, use either:

  • Environment variables (DJ_USER, DJ_PASS, DJ_HOST, etc.)
  • The .secrets/ directory (created alongside datajoint.json)

Parameters:

Name Type Description Default
path str or Path

Where to save the template. Default 'datajoint.json'.

 'datajoint.json'
minimal bool

If True (default), create minimal template with just database settings. If False, create full template with all available settings.

 True
create_secrets_dir bool

If True (default), also create a .secrets/ directory with template files for credentials.

 True

Returns:

Type Description
Path

Absolute path to the created config file.

Raises:

Type Description
FileExistsError

If config file already exists (won't overwrite).

Examples:

>>> import datajoint as dj
>>> dj.config.save_template()  # Creates minimal template + .secrets/
>>> dj.config.save_template("full-config.json", minimal=False)

get 

get(key, default=None)

Get setting with optional default value.