Architecture

Package Structure

bc_dev_tools is a Python package invoked as python -m bc_dev_tools. The package lives within the larger Bestway BC Development repository and depends on shared API clients in the repository's tools/ directory.

<repo-root>/
  .env                          # Shared BC OAuth2 credentials
  tools/
    bc_odata_client.py          # BC OData API client
    faq_api_client.py           # FAQ API client
    csm_1_api_client.py         # CSM 1.0 API client
  bc_dev_tools/
    __init__.py
    __main__.py                 # Entry point
    cli.py                      # CLI dispatcher
    main.py                     # Interactive menu + handler functions
    config.py                   # Centralized configuration
    database_manager.py         # SQLite + pandas utilities
    exporter.py                 # Excel export
    faqbc_data_matcher.py       # BC/FAQ fuzzy matching
    bc_data_normalizer.py       # BC normalization
    faq_data_normalizer.py      # FAQ normalization
    failed_csm_resolver.py      # Resolver pipeline
    analyze_resolver.py         # Resolver pattern analysis
    analyze_human_corrections.py # Correction pattern analysis
    test_faq_auth.py            # FAQ auth debug script
    .env                        # Local overrides (not committed)
    .env.example

Entry Point Chain

When you run python -m bc_dev_tools, Python executes __main__.py. That module performs three setup tasks before handing off to the CLI:

Load environment variables — python-dotenv loads the repository root .env first (shared BC OAuth2 credentials), then the local bc_dev_tools/.env with override=True. This layered approach lets the package override specific variables without duplicating the shared ones.
Configure sys.path — The shared API clients (bc_odata_client, faq_api_client, csm_1_api_client) live in the repository's tools/ directory, which is not a Python package. __main__.py inserts that directory into sys.path so downstream modules can import them directly. The path defaults to <repo-root>/tools/ but can be overridden with the BC_TOOLS_DIR environment variable.
Dispatch to cli.main() — After setup, __main__.py calls cli.main(), which parses arguments and routes to the appropriate handler.

python -m bc_dev_tools [args]
        |
   __main__.py
        |--- load_dotenv(<repo-root>/.env)
        |--- load_dotenv(bc_dev_tools/.env, override=True)
        |--- sys.path.insert(0, tools/)
        |
   cli.main(sys.argv)
        |
   argparse dispatch
        |
   +--- pipeline ------> main.complete_pipeline()
   +--- compare -------> main.compare_data_cli()
   +--- export --------> main.export_results_cli()
   +--- submit --------> main.submit_failed_quality_reports_cli()
   +--- interactive ----> main.main() [menu loop]
   +--- import faq -----> main.import_faq_api_with_normalization()
   +--- import bc ------> main.import_bc_fault_codes_cli()
   +--- import excel ---> main.import_excel_cli()
   +--- import manual-report -> main.import_manual_report_compilation()
   +--- resolve run ----> failed_csm_resolver.main()
   +--- resolve analyze -> analyze_resolver.analyze()
   +--- resolve corrections -> analyze_human_corrections.analyze()

CLI Dispatch (`cli.py`)

cli.py defines the argparse tree and dispatches to handler functions. It uses lazy imports inside each command handler to avoid loading heavy dependencies (pandas, numpy, API clients) at parse time. This keeps --help fast and avoids import errors when credentials are not set.

The argparse tree has three levels:

Top-level commands: pipeline, compare, export, submit, interactive
Import group: import faq, import bc, import excel, import manual-report
Resolve group: resolve run, resolve analyze, resolve corrections

Module Responsibilities

Module	Purpose
`__main__.py`	Environment setup and entry point
`cli.py`	Argument parsing and command dispatch
`main.py`	Interactive menu loop, orchestration functions, and non-interactive CLI wrappers
`config.py`	Reads environment variables, defines table names, column mappings, default thresholds
`database_manager.py`	SQLite connection management, Excel/API imports, DataFrame-to-table creation, querying
`faqbc_data_matcher.py`	Multi-column fuzzy matching using `difflib.SequenceMatcher`
`exporter.py`	Writes comparison results to multi-sheet Excel workbooks via `openpyxl`
`bc_data_normalizer.py`	Breaks flat BC data into dimension tables (fault areas, symptom codes, fault codes, descriptions) and creates SQL views
`faq_data_normalizer.py`	Breaks flat FAQ data into dimension tables (main categories, sub categories, issue types, item parts) and creates SQL views
`failed_csm_resolver.py`	End-to-end pipeline: fetch from BC OData, match against FAQ, write spreadsheet, optionally submit to CSM and update BC
`analyze_resolver.py`	Statistical analysis of resolver output patterns
`analyze_human_corrections.py`	Analysis of manual corrections made to resolver output
`test_faq_auth.py`	Standalone debug script for testing FAQ API authentication

Shared Dependencies (`tools/`)

The package imports three API clients from the repository's tools/ directory:

bc_odata_client.py — OAuth2 client credentials flow against Azure AD, CRUD operations on BC OData endpoints. Used by failed_csm_resolver.py.
faq_api_client.py — Username/password authentication against the FAQ API, retrieves issue type hierarchies. Used by database_manager.py and failed_csm_resolver.py.
csm_1_api_client.py — Secret-based authentication against the CSM 1.0 API, submits quality reports. Used by main.py and failed_csm_resolver.py.

These clients are shared across multiple packages in the repository, which is why they live outside bc_dev_tools/.

Dual-Mode Execution

main.py supports two import strategies to enable both package mode and standalone execution:

try:
    from bc_dev_tools.config import ...
except ImportError:
    from config import ...

Package mode (python -m bc_dev_tools): Uses absolute imports (bc_dev_tools.config).
Standalone mode (python main.py from within the directory): Falls back to relative imports. This preserves backwards compatibility for existing scripts and workflows.

Data Flow

Import and Compare Pipeline

FAQ API  ──> faq_api_client ──> database_manager ──> SQLite
                                                       |
BC Excel ──> database_manager ─────────────────────> SQLite
                                                       |
                          faq_data_normalizer ──> normalized tables + views
                          bc_data_normalizer  ──> normalized tables + views
                                                       |
                          faqbc_data_matcher  ──> match columns on BC table
                                                       |
                          exporter ──> comparison_results.xlsx

Failed CSM Resolver Pipeline

BC OData API ──> bc_odata_client ──> failed service orders
                                          |
FAQ API ──> faq_api_client ──> issue type hierarchy
                                          |
                   failed_csm_resolver ──> fuzzy match + spreadsheet write
                                          |
                   (optional) ──> csm_1_api_client ──> CSM submission
                   (optional) ──> bc_odata_client  ──> mark sent in BC

Data Model

Source tables — raw imports from API responses and Excel files, stored as-is in SQLite
Normalized FAQ tables — faq_main_categories, faq_sub_categories, faq_issue_types, faq_item_parts, faq_issue_types_normalized (fact table), plus SQL views joining them
Normalized BC tables — bc_fault_areas, bc_symptom_codes, bc_fault_codes_dim, bc_descriptions, bc_issue_types_normalized (fact table), plus SQL views joining them
Match augmentation — columns appended to the BC source table to store match results (faq_match_found, faq_matched_id, faq_match_details, match_score)

Package Structure​

Entry Point Chain​

CLI Dispatch (cli.py)​

Module Responsibilities​

Shared Dependencies (tools/)​

Dual-Mode Execution​

Data Flow​

Import and Compare Pipeline​

Failed CSM Resolver Pipeline​

Data Model​