[Refactor] Modernize configuration management using Pydantic Settings

[sushant-suse]

Situation

The current configuration loading logic in src/docbuild/config/load.py utilizes manual dictionary merging (deep_merge) and a custom file-discovery loop. While functional, this "low-level" approach is difficult to maintain and expand as we add more configuration parameters.

Proposed Changes:

1. Migrate to Pydantic-Settings: Replace manual TOML loading with a BaseSettings class.
2. Automated Search Path: Use Pydantic's SettingsSource to automatically look for docbuild.toml in:
   • /etc/docbuild/
   • ~/.config/docbuild/
   • Current project root
3. Unified Validation: Ensure that CLI overrides (like --max-workers) and Environment Variables (like DOCBUILD_DEBUG) are validated by the same model as the TOML file.
4. Simplify load.py: Reduce the custom code in load.py to a single "factory" call that returns a fully validated AppConfig object.

Use Case

• Reduces boilerplate code in the CLI and config loader.
• Provides better error messages when a user provides an invalid setting.
• Enables easier "merging" logic where environment variables automatically override TOML settings.

Possible Implementation

No response