Initial commit

.env.example

@@ -0,0 +1,18 @@
+# Gemini API key used by Weaviate's Gemini vectorizer backend and the Gemini LLM provider
+GEMINI_API_KEY=your-gemini-api-key
+
+# Optional LLM provider keys (required when config.yaml llm.provider is openai / anthropic)
+OPENAI_API_KEY=your-openai-api-key
+ANTHROPIC_API_KEY=your-anthropic-api-key
+
+# Weaviate connection (local Docker instance)
+WEAVIATE_HOST=localhost
+WEAVIATE_HTTP_PORT=8080
+WEAVIATE_GRPC_PORT=50051
+
+# Microsoft SQL Server connection
+SQL_DRIVER=ODBC Driver 17 for SQL Server
+SQL_SERVER=localhost
+SQL_DATABASE=your-database-name
+SQL_USERNAME=your-sql-username
+SQL_PASSWORD=your-sql-password

.gitignore

@@ -0,0 +1,12 @@
+.venv/
+.env
+output/
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.DS_Store
+Thumbs.db

README.md

@@ -0,0 +1,254 @@
+# AI Agro Support — Daily Pipeline
+
+Daily agronomic advisory pipeline. For every field growing a configured crop, it
+builds an 11-day weather / phenology / disease-forecast JSON, asks an LLM to
+synthesise one or two vector-search queries, prefilters chemical products in SQL
+Server, and retrieves up to six matching products from a local Weaviate
+`ProductProfile` collection. It then enriches that JSON with full product label
+information and the previous advisory, asks a second LLM to write the
+farmer-facing advice, and stores the result in the `advice` table.
+
+The pipeline is **crop-first**: `config.yaml` declares which crops and diseases
+to check, and each morning's `batch` run discovers every field that grows one of
+those crops and has a matching disease-forecast model configured, then runs the
+full sequence above for each `(field, disease)` pair — in parallel, with one
+field's failure never blocking another's.
+
+## Prerequisites
+
+- Python 3.12+
+- Microsoft SQL Server (`DBMeteo`) reachable with ODBC Driver 17 (or 18)
+- Weaviate running locally (see `docker-compose.yml`) with a populated `ProductProfile` collection
+- API key for the chosen LLM provider (Gemini by default)
+
+## Setup
+
+```powershell
+cd d:\Uni\Thesis\AI-Agro-Support
+python -m venv .venv
+.\.venv\Scripts\Activate.ps1
+pip install -r requirements.txt
+copy .env.example .env
+# Edit .env with SQL, Weaviate, and LLM credentials
+```
+
+Start Weaviate if it is not already running:
+
+```powershell
+docker compose up -d
+```
+
+## Configuration
+
+- [`config.yaml`](config.yaml) — the `crops:` worklist, concurrency/rate limits, retry policy,
+  the 09:00 SLA deadline, LLM provider/model, and vocab paths
+- [`.env`](.env) — SQL Server, Weaviate, and API keys
+- [`vocab/crops.yaml`](vocab/crops.yaml) / [`vocab/diseases.yaml`](vocab/diseases.yaml) — Italian → English maps
+- [`prompts/`](prompts) — the prompt registry (see below)
+
+### `crops:` worklist
+
+```yaml
+crops:
+  - crop: grapevine              # canonical English, matched via vocab/crops.yaml
+    diseases:
+      - model_name: PERONOSPORA  # matched against AI_agrosupport_agro_models.anmod_model
+        disease: downy mildew    # canonical English, used for products/advice/prompts
+```
+
+Every morning, `batch` maps each `crop` to the fields growing it
+(`AI_agrosupport_an_colture` → `AI_agrosupport_cmp_layers`), joins to
+`AI_agrosupport_agro_models` for each listed `model_name`, and runs one job per
+matching `(field, disease)` pair. To add a crop or disease: add an entry here, add
+the Italian/English mapping to `vocab/crops.yaml` / `vocab/diseases.yaml` if it
+doesn't already exist, and add a prompt directory (see below) — otherwise the
+pair silently uses the generic fallback prompt.
+
+### `worklist:`, `concurrency:`, `retry:`, `schedule:`
+
+```yaml
+worklist:
+  require_enabled_model: true   # AI_agrosupport_agro_models.anmod_enabled must be 1
+  field_allowlist: []           # restrict to specific field IDs while testing
+
+concurrency:
+  workers: 8                    # ThreadPoolExecutor size
+  limits: { sql: 6, gemini: 4 } # see "Concurrency model" below
+  gemini_requests_per_minute: 60
+
+retry:
+  attempts: 3
+  initial_backoff_seconds: 2
+  max_backoff_seconds: 30
+
+schedule:
+  deadline: "09:00"             # jobs not started by this local time are skipped
+```
+
+### Prompt registry (`prompts/`)
+
+Each crop-disease pair needs its own advice prompt (its phytopathological
+model's phase semantics, phenology notes, etc.). Prompts are plain Markdown
+files resolved by `pipeline/prompts.py::PromptRegistry`, with a shared fallback
+so a newly configured pair never fails outright — it just logs a warning and
+uses a generic prompt until a dedicated one is written.
+
+```
+prompts/
+  _output_format.md                          shared JSON contract, appended to every advice system prompt
+  query_synthesis/_default.md                generic query-synthesis system prompt
+  query_synthesis/<crop>__<disease>.md        optional per-pair override
+  advice/_default/{system.md,user.md}         fallback advice prompts
+  advice/<crop>__<disease>/{system.md,user.md} per-pair advice prompts
+```
+
+The `<crop>__<disease>` slug is built from the same canonical English names used
+in `config.yaml`'s `crops:` block (lower-cased, non-alphanumeric runs collapsed
+to `_`) — e.g. `grapevine` + `downy mildew` → `grapevine__downy_mildew`. To add a
+prompt for a new pair, create `prompts/advice/<slug>/system.md` and `user.md`;
+`batch` logs every configured pair still falling back to `_default` at startup.
+
+## Run
+
+```powershell
+.\.venv\Scripts\Activate.ps1
+
+# Daily entry point: every crop/disease pair in config.yaml, across all matching fields
+python -m pipeline batch
+
+# Historical / test dates (useful when the live window has no FASE2/FASE5)
+python -m pipeline batch --as-of 2026-06-29
+
+# Narrow to one crop, disease, and/or field while testing
+python -m pipeline batch --crop grapevine --disease "downy mildew" --field-id 4085
+
+# Re-run a partially failed morning without redoing already-written advice
+python -m pipeline batch --skip-existing-advice
+
+# Build every job's JSON but skip LLM calls, vector search, and the DB write
+python -m pipeline batch --dry-run
+
+# Single-field debugging (keeps the pre-scaling CLI behaviour)
+python -m pipeline one --field-id 4085 --disease PERONOSPORA --dry-run
+```
+
+`batch` accepts `--deadline HH:MM` and `--workers N` to override
+`schedule.deadline` / `concurrency.workers` from `config.yaml` for one run.
+
+## Concurrency model
+
+One `ThreadPoolExecutor` (`concurrency.workers`) runs every discovered job.
+Because dozens to a couple hundred jobs can run in the same morning, several
+things that used to be "open it, use it, close it" per field are now shared
+across the whole run (see `pipeline/resources.py`):
+
+- **SQL**: each worker thread gets its own lazily-created `pyodbc` connection
+  (connections cannot be shared across threads), kept open for the batch's
+  lifetime. `concurrency.limits.sql` bounds how many jobs may run their
+  SQL-heavy phase (forecast/weather/phenology/treatments lookup, or the
+  product-JSON/advice-insert phase) at the same instant, independent of how
+  many connections are open.
+- **Weaviate**: one client is opened for the whole batch instead of one per
+  field.
+- **Gemini**: `near_text` search embeds through the same `GEMINI_API_KEY` as
+  both LLM calls (`ProductProfile` is vectorised with text2vec-palm /
+  gemini-embedding-001), so `concurrency.limits.gemini` and
+  `gemini_requests_per_minute` are a single shared budget covering query
+  synthesis, advice generation, *and* vector search — not two independent
+  ones.
+- **Products**: the whole `products` table is loaded into memory once per
+  batch (`pipeline/stages/products.py::ProductIndex`) instead of re-scanned
+  and re-parsed once per field, and `ProductJsonBuilder` is shared per
+  `(crop, disease)` pair instead of per field.
+
+Each of the three external boundaries — LLM, Weaviate, SQL — has its own
+`tenacity` retry policy in `pipeline/retry.py` (exponential backoff with
+jitter, `retry.attempts` tries), applied automatically inside
+`Resources.call_llm` / `Resources.search_products` and around the
+`insert_advice` transaction.
+
+## Output
+
+Each run writes to `output/<YYYY-MM-DD>/`:
+
+- `field<field_id>__<crop>__<disease slug>.json` — one file per job (see fields below)
+- `_run_report.json` — the operational artifact for the morning: totals by
+  status plus a per-job breakdown (`status`, `duration_s`,
+  `recommended_count`, `allowed_products_empty`, `message`, and, on failure,
+  `error_type` / `error_message`)
+
+Per-job payload fields:
+
+| Field | Description |
+|---|---|
+| `run` | Metadata (as_of, field, crop, disease, organic, station, …) |
+| `json_for_advice_generation` | Full 11-day meteorological + forecast payload, plus `allowed_products`, `applied_treatment` and `last_advice`; this is what the advice LLM receives |
+| `json_for_query_synthesis` | Subset of the above, sent to the LLM for search query generation |
+| `generated_queries` | One or two natural-language search queries |
+| `candidate_count` | Products remaining after the prefilter |
+| `recommended_products` | Up to 6 items with `product_name` + `registration_number` |
+| `last_advice` | `advice_summary` + `suggested_products` of the most recent previous advisory |
+| `advice` | Parsed LLM output: `full_advice`, `advice_summary`, `apply_treatment`, `treatments`, `dosage`, `apply_date` |
+| `sent_information` | The advice payload with every embedded product label collapsed back to a product name; mirrors `advice.sent_information` |
+| `advice_inserted` | Whether the `advice` row was written |
+| `status` | `ok`, `no_risk`, or `dry_run` (see `_run_report.json` for `failed` / `skipped_*`, which have no per-job file) |
+| `message` | Human-readable summary of the run outcome |
+
+If neither `FASE2` nor `FASE5` appears in a field's 11-day window, the product
+search is skipped and `allowed_products` is empty, but advice is still generated
+and stored; that job is labelled `no_risk`. The run report also counts `ok` jobs
+whose `allowed_products` ended up empty anyway (for example an organic field
+with no organic-certified product in the catalog yet) — watch that number as the
+`products` table fills in, since a sudden jump usually means the prefilter or
+the Weaviate allowlist filter is misbehaving rather than that the catalog is
+genuinely empty for that combination.
+
+The process exit code is `0` only if every job in the report is `ok`, `no_risk`,
+`dry_run`, or a deliberate `skipped_*`; it is `1` if any job's status is `failed`.
+
+## Database writes
+
+Each non-dry-run job inserts one row into `advice` for `(date, field_id,
+disease)`. That triple is the table's primary key, so any existing row for
+the same triple is deleted first — re-running a day replaces its advisory
+instead of failing on a duplicate key. The delete-then-insert pair runs
+inside one transaction (`pipeline/db.py::transaction`) so a crash between
+the two statements cannot leave a field's advisory missing for the day, and
+deadlocks are retried per `retry:` in config.yaml.
+
+The primary key's clustered index on `([date], field_id, disease)` also
+lets that DELETE seek straight to the rows it needs instead of scanning the
+table, so concurrent workers don't block each other on unrelated rows.
+
+## Daily scheduling
+
+Provide a CLI entry point only. Schedule it yourself on the deployment machine.
+Advice must be ready by the `schedule.deadline` (09:00 by default), and weather
+data should stay fresh, so start the run no earlier than ~07:00 — for example
+via Windows Task Scheduler, daily at 07:00:
+
+```
+d:\Uni\Thesis\AI-Agro-Support\.venv\Scripts\python.exe -m pipeline batch --skip-existing-advice
+```
+
+with working directory `d:\Uni\Thesis\AI-Agro-Support`. `--skip-existing-advice`
+makes a re-trigger (e.g. a scheduled retry later that morning) resumable at no
+extra cost: it does one bulk check against `advice` up front and only runs jobs
+that don't have a row yet.
+
+## Pipeline stages
+
+Run once per `(field, disease)` job by `pipeline/job.py::run_job`:
+
+1. **Worklist** — crop → fields (`AI_agrosupport_an_colture` + `AI_agrosupport_cmp_layers`) → disease model (`AI_agrosupport_agro_models`); organic flag is `cmplay_imp == 3` (`pipeline/worklist.py`, run once per batch, not per job)
+2. **Disease model** — per-day `FASE5` > `FASE2` resolution over as_of±5, from the worklist's known `anmod_id`
+3. **Weather** — `TDatiMeteo_D` for past days, `TDatiMeteo_D_FRC` for today/future
+4. **Phenology** — carry-forward of latest observation ≤ day; future days null
+5. **Applied treatments** — products sprayed over `as_of-5 .. as_of`, from `AI_agrosupport_agro_ril_operations*` with `rilop_operation = 10`
+6. **LLM** — synthesise 1–2 anonymised search queries from the subset
+7. **Product prefilter** — organic / crop / disease / FASE rules against the batch-wide, in-memory `products` index
+8. **Vector search** — Weaviate `near_text` on `ProductProfile`, filtered server-side to the allowlisted `product_id`s
+9. **Product JSON** — label details per product from `products`, `product_uses`, `label_chunks`, cached per `(crop, disease)`
+10. **Context enrichment** — `allowed_products`, `applied_treatment` expansion, `last_advice` from the `advice` table
+11. **Advice generation** — second LLM profile returns the structured advisory
+12. **Persistence** — the advisory plus its collapsed input are written to `advice` inside one transaction

config.yaml

@@ -0,0 +1,90 @@
+# ── LLM configuration ─────────────────────────────────────────────────────────
+# Two independent profiles, one per pipeline stage. Each accepts the same keys:
+#   provider: openai | anthropic | gemini
+#   openai_model / anthropic_model / gemini_model / max_tokens / temperature
+llm:
+  # Part_One: vector-search query synthesis (small structured JSON in,
+  # 1-2 short queries out). A lighter/cheaper model is sufficient here.
+  query_synthesis:
+    provider: gemini
+    openai_model: gpt-4o-mini
+    anthropic_model: claude-haiku-4-5
+    gemini_model: gemini-2.5-flash
+    max_tokens: 8192
+    temperature: 0.0
+
+  # Part_Two: the farmer-facing advisory generator. Receives the full
+  # json_for_advice_generation payload (weather, phenology, model output,
+  # applied treatments, product labels) and returns the structured advice
+  # JSON stored in the advice table. Needs a more capable model.
+  advice_generation:
+    provider: gemini
+    openai_model: gpt-4o
+    anthropic_model: claude-opus-4-5
+    gemini_model: gemini-2.5-pro
+    max_tokens: 16384
+    temperature: 0.0
+
+# ── Crop-first worklist ────────────────────────────────────────────────────────
+# `python -m pipeline batch` discovers jobs from this list: for every crop, it
+# finds all fields growing it (AI_agrosupport_cmp_layers ⨝ an_colture) that also
+# have a matching row in AI_agrosupport_agro_models for each listed disease's
+# model_name, and runs the full single-field pipeline for every resulting
+# (field, disease) pair. `disease` is the canonical English name used for the
+# product prefilter, the vector search, and the advice/prompt lookup; it must
+# exist in vocab/diseases.yaml. `model_name` is matched (case-insensitively)
+# against AI_agrosupport_agro_models.anmod_model.
+#
+# Each crop-disease pair needs a matching prompt directory under
+# prompts/advice/<crop>__<disease slug>/ (see prompts/README or the pipeline
+# README for the exact slug rules); pairs without one fall back to
+# prompts/advice/_default/ and are logged at startup.
+crops:
+  - crop: grapevine
+    diseases:
+      - model_name: PERONOSPORA
+        disease: downy mildew
+
+# ── Worklist filters ───────────────────────────────────────────────────────────
+worklist:
+  # AI_agrosupport_agro_models.anmod_enabled is True for every model row in the
+  # current database; keep this on so a disabled model never generates advice.
+  require_enabled_model: true
+  # Restrict the batch to specific field IDs while testing; empty = no filter.
+  field_allowlist: []
+
+# ── Concurrency & rate limiting ────────────────────────────────────────────────
+# One "gemini" budget covers both direct LLM calls (query synthesis, advice
+# generation) and Weaviate near_text search, because the ProductProfile
+# collection is vectorised with text2vec-palm (gemini-embedding-001) using the
+# same GEMINI_API_KEY — they share one quota.
+concurrency:
+  workers: 8
+  limits:
+    sql: 6
+    gemini: 4
+  gemini_requests_per_minute: 60
+
+# ── Retry policy for transient LLM / Weaviate / SQL failures ──────────────────
+retry:
+  attempts: 3
+  initial_backoff_seconds: 2
+  max_backoff_seconds: 30
+
+# ── Daily SLA ───────────────────────────────────────────────────────────────────
+# The batch must not still be running after this local time; jobs not yet
+# started by then are skipped (status skipped_deadline) rather than risking a
+# late advisory. Schedule the run itself no earlier than ~07:00 so weather is
+# fresh (see README).
+schedule:
+  deadline: "09:00"
+
+# ── Single-field debug mode (`python -m pipeline one`) ────────────────────────
+# Not used by `batch`; kept for ad-hoc single-field runs and debugging.
+field_id: 4012
+disease_name: "PERONOSPORA"
+
+# ── Vocabulary paths ──────────────────────────────────────────────────────────
+vocab:
+  crops: vocab/crops.yaml
+  diseases: vocab/diseases.yaml

docker-compose.yml

@@ -0,0 +1,26 @@
+services:
+  weaviate:
+    command:
+      - --host
+      - 0.0.0.0
+      - --port
+      - "8080"
+      - --scheme
+      - http
+    image: cr.weaviate.io/semitechnologies/weaviate:1.32.0 # Required for text2vec-google taskType support
+    ports:
+      - 8080:8080
+      - 50051:50051
+    volumes:
+      - weaviate_data:/var/lib/weaviate
+    restart: on-failure:0
+    environment:
+      QUERY_DEFAULTS_LIMIT: 25
+      PERSISTENCE_DATA_PATH: "/var/lib/weaviate"
+      CLUSTER_HOSTNAME: "node1"
+      # Swap OpenAI to Google modules
+      ENABLE_MODULES: "text2vec-google"
+      DEFAULT_VECTORIZER_MODULE: "text2vec-google"
+
+volumes:
+  weaviate_data:

pipeline/__init__.py

@@ -0,0 +1 @@
+"""AI Agro Support daily data-processing pipeline."""

pipeline/__main__.py

@@ -0,0 +1,221 @@
+"""CLI entry point.
+
+    python -m pipeline batch [--as-of DATE] [--crop NAME] [--disease NAME]
+                              [--field-id N ...] [--deadline HH:MM]
+                              [--skip-existing-advice] [--dry-run] [--workers N]
+
+    python -m pipeline one --field-id N --disease NAME [--as-of DATE] [--dry-run]
+
+`batch` is the daily entry point: it discovers every (field, disease) job
+implied by config.yaml's `crops:` list and the database, and runs them
+through a bounded thread pool (see pipeline/batch.py). `one` keeps the
+original single-field behaviour for ad-hoc debugging.
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import sys
+from datetime import date, datetime
+
+from pipeline.batch import run_batch
+from pipeline.config import load_settings
+from pipeline.errors import NoModelConfiguredError, PipelineError
+from pipeline.job import run_job
+from pipeline.output import write_job_output
+from pipeline.prompts import PromptRegistry
+from pipeline.resources import Resources
+from pipeline.stages.disease import find_anmod_id
+from pipeline.stages.field import resolve_field
+from pipeline.vocab import load_crop_vocab, load_disease_vocab
+from pipeline.worklist import Job
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
+)
+logger = logging.getLogger("pipeline")
+
+
+def _parse_as_of(value: str | None) -> date:
+    if value is None:
+        return date.today()
+    try:
+        return datetime.strptime(value, "%Y-%m-%d").date()
+    except ValueError as exc:
+        raise argparse.ArgumentTypeError(
+            f"Invalid --as-of date '{value}'. Expected YYYY-MM-DD."
+        ) from exc
+
+
+def _run_one(args: argparse.Namespace) -> int:
+    """`python -m pipeline one`: the original single-field pipeline, now built
+    on top of `run_job` so it stays behaviourally identical to `batch`."""
+    settings = load_settings(field_id_override=args.field_id, disease_name_override=args.disease)
+    if settings.field_id is None or settings.disease_name is None:
+        raise PipelineError(
+            "`one` needs --field-id and --disease "
+            "(or field_id / disease_name set in config.yaml)."
+        )
+
+    crop_vocab = load_crop_vocab(settings.crops_vocab)
+    disease_vocab = load_disease_vocab(settings.diseases_vocab)
+    disease_english = disease_vocab.to_english(settings.disease_name, kind="disease")
+
+    logger.info(
+        "Starting single-field run as_of=%s field_id=%s disease=%s (%s) dry_run=%s",
+        args.as_of.isoformat(),
+        settings.field_id,
+        settings.disease_name,
+        disease_english,
+        args.dry_run,
+    )
+
+    prompts = PromptRegistry(settings.prompts_dir)
+    resources = Resources(settings, prompts)
+    try:
+        conn = resources.sql_connection()
+        field = resolve_field(conn, settings.field_id, crop_vocab)
+        anmod_id = find_anmod_id(conn, settings.field_id, settings.disease_name)
+        resources.load_product_index()
+
+        job = Job(
+            field_id=settings.field_id,
+            crop_english=field.crop_english,
+            crop_italian=field.crop_italian,
+            model_name=settings.disease_name,
+            disease_english=disease_english,
+            organic=field.organic,
+            station=field.cmplay_station,
+            anmod_id=anmod_id,
+        )
+        result = run_job(resources, job, args.as_of, dry_run=args.dry_run)
+    finally:
+        resources.close()
+
+    if result.payload is not None:
+        path = write_job_output(settings.output_dir, args.as_of, result.payload)
+        logger.info("Wrote output -> %s", path)
+
+    if result.status == "failed":
+        logger.error("%s: %s", result.error_type, result.error_message)
+        return 1
+    logger.info("Done: status=%s message=%s", result.status, result.message)
+    return 0
+
+
+def _run_batch(args: argparse.Namespace) -> int:
+    """`python -m pipeline batch`: the daily crop-first multi-field run."""
+    settings = load_settings()
+    crop_vocab = load_crop_vocab(settings.crops_vocab)
+
+    logger.info(
+        "Starting batch as_of=%s crop_filter=%s disease_filter=%s field_filter=%s "
+        "skip_existing_advice=%s dry_run=%s workers=%s",
+        args.as_of.isoformat(),
+        args.crop,
+        args.disease,
+        args.field_id,
+        args.skip_existing_advice,
+        args.dry_run,
+        args.workers or settings.concurrency.workers,
+    )
+
+    return run_batch(
+        settings,
+        crop_vocab,
+        args.as_of,
+        crop=args.crop,
+        disease=args.disease,
+        field_ids=args.field_id,
+        deadline_override=args.deadline,
+        skip_existing_advice=args.skip_existing_advice,
+        dry_run=args.dry_run,
+        workers_override=args.workers,
+    )
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description="Daily agronomic advice pipeline.")
+    subparsers = parser.add_subparsers(dest="command")
+
+    batch_parser = subparsers.add_parser(
+        "batch",
+        help="Run every configured crop/disease pair across all matching fields.",
+    )
+    batch_parser.add_argument(
+        "--as-of", type=_parse_as_of, default=None,
+        help="Reference date (YYYY-MM-DD). Defaults to today.",
+    )
+    batch_parser.add_argument(
+        "--crop", default=None, help="Only run this crop (canonical English name).",
+    )
+    batch_parser.add_argument(
+        "--disease", default=None, help="Only run this disease (canonical English name).",
+    )
+    batch_parser.add_argument(
+        "--field-id", type=int, action="append", default=None,
+        help="Only run this field ID (repeatable).",
+    )
+    batch_parser.add_argument(
+        "--deadline", default=None,
+        help="Override schedule.deadline from config.yaml (HH:MM, local time).",
+    )
+    batch_parser.add_argument(
+        "--skip-existing-advice", action="store_true",
+        help="Skip (field, disease) pairs that already have an advice row for --as-of.",
+    )
+    batch_parser.add_argument(
+        "--dry-run", action="store_true",
+        help="Build the JSON for every job but skip LLM calls, vector search, and the advice DB write.",
+    )
+    batch_parser.add_argument(
+        "--workers", type=int, default=None, help="Override concurrency.workers from config.yaml.",
+    )
+    batch_parser.set_defaults(func=_run_batch)
+
+    one_parser = subparsers.add_parser(
+        "one",
+        help="Run a single field/disease pair (debugging; the original single-field CLI behaviour).",
+    )
+    one_parser.add_argument(
+        "--as-of", type=_parse_as_of, default=None,
+        help="Reference date (YYYY-MM-DD). Defaults to today.",
+    )
+    one_parser.add_argument(
+        "--field-id", type=int, default=None, help="Field ID (defaults to config.yaml's field_id).",
+    )
+    one_parser.add_argument(
+        "--disease", default=None,
+        help="Disease model name, e.g. PERONOSPORA (defaults to config.yaml's disease_name).",
+    )
+    one_parser.add_argument(
+        "--dry-run", action="store_true",
+        help="Build the JSON but skip LLM calls, vector search, and the advice DB write.",
+    )
+    one_parser.set_defaults(func=_run_one)
+
+    args = parser.parse_args(argv)
+    if not args.command:
+        parser.print_help()
+        return 2
+
+    if not isinstance(args.as_of, date):
+        args.as_of = _parse_as_of(None)
+
+    try:
+        return args.func(args)
+    except NoModelConfiguredError as exc:
+        logger.error("%s", exc)
+        return 2
+    except PipelineError as exc:
+        logger.error("%s", exc)
+        return 1
+    except Exception:
+        logger.exception("Unhandled pipeline failure")
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

pipeline/batch.py

@@ -0,0 +1,196 @@
+"""Batch orchestrator: discover jobs, run them across a bounded thread pool,
+and write one run report per day.
+
+One `ThreadPoolExecutor` runs every discovered (field, disease) job so a slow
+crop cannot serialise the others; `pipeline.job.run_job` isolates each job's
+failures from the rest. This module owns everything above that: worklist
+discovery + CLI filters, the `--skip-existing-advice` resume check, the
+09:00-style deadline, and writing the run report.
+"""
+
+from __future__ import annotations
+
+import logging
+import threading
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from datetime import date, datetime
+from datetime import time as dtime
+
+from pipeline.config import Settings
+from pipeline.db import fetch_all
+from pipeline.job import JobResult, run_job
+from pipeline.output import write_job_output, write_run_report
+from pipeline.prompts import PromptRegistry
+from pipeline.resources import Resources
+from pipeline.vocab import Vocabulary
+from pipeline.worklist import Job, build_jobs
+
+logger = logging.getLogger(__name__)
+
+_EXISTING_ADVICE_SQL = "SELECT field_id, disease FROM advice WHERE [date] = ?"
+
+
+def _parse_deadline(deadline_str: str) -> datetime:
+    """
+    Build today's deadline datetime from an "HH:MM" string.
+
+    Deliberately anchored to the wall-clock date the batch is actually
+    running on, not `as_of`: the SLA ("advice ready by 09:00") is about when
+    the run happens, and in production `as_of` is always today anyway.
+    Anchoring to `as_of` instead would make every historical/backfill run
+    (`--as-of` in the past) look like its deadline had already passed.
+    """
+    try:
+        hour_str, minute_str = deadline_str.split(":", 1)
+        return datetime.combine(date.today(), dtime(hour=int(hour_str), minute=int(minute_str)))
+    except (ValueError, TypeError) as exc:
+        raise ValueError(f"Invalid deadline '{deadline_str}'. Expected HH:MM.") from exc
+
+
+def filter_jobs(
+    jobs: list[Job],
+    *,
+    crop: str | None = None,
+    disease: str | None = None,
+    field_ids: list[int] | None = None,
+) -> list[Job]:
+    """Narrow the discovered worklist to CLI-selected crop / disease / field IDs."""
+    filtered = jobs
+    if crop:
+        filtered = [j for j in filtered if j.crop_english.lower() == crop.strip().lower()]
+    if disease:
+        filtered = [j for j in filtered if j.disease_english.lower() == disease.strip().lower()]
+    if field_ids:
+        wanted = set(field_ids)
+        filtered = [j for j in filtered if j.field_id in wanted]
+    return filtered
+
+
+def _existing_advice_keys(resources: Resources, as_of: date) -> set[tuple[int, str]]:
+    """One bulk SELECT instead of one per job, for --skip-existing-advice."""
+    conn = resources.sql_connection()
+    rows = fetch_all(conn, _EXISTING_ADVICE_SQL, (as_of,))
+    return {(int(row.field_id), str(row.disease)) for row in rows}
+
+
+def run_batch(
+    settings: Settings,
+    crop_vocab: Vocabulary,
+    as_of: date,
+    *,
+    crop: str | None = None,
+    disease: str | None = None,
+    field_ids: list[int] | None = None,
+    deadline_override: str | None = None,
+    skip_existing_advice: bool = False,
+    dry_run: bool = False,
+    workers_override: int | None = None,
+) -> int:
+    """Run every discovered job and write the run report. Returns the process exit code."""
+    prompts = PromptRegistry(settings.prompts_dir)
+    configured_pairs = [(cp.crop, dp.disease) for cp in settings.crops for dp in cp.diseases]
+    fallbacks = prompts.validate(configured_pairs)
+    if fallbacks:
+        logger.warning(
+            "%d crop/disease pair(s) have no dedicated advice prompt and will fall back "
+            "to prompts/advice/_default/: %s",
+            len(fallbacks),
+            fallbacks,
+        )
+
+    resources = Resources(settings, prompts)
+    try:
+        jobs = build_jobs(resources.sql_connection(), settings.crops, settings.worklist, crop_vocab)
+        jobs = filter_jobs(jobs, crop=crop, disease=disease, field_ids=field_ids)
+        logger.info("Worklist discovered %d job(s) for as_of=%s", len(jobs), as_of.isoformat())
+
+        if not jobs:
+            logger.warning(
+                "No jobs to run for as_of=%s; nothing matched the configured "
+                "crops/diseases (or the CLI filters).",
+                as_of.isoformat(),
+            )
+            write_run_report(settings.output_dir, as_of, [])
+            return 0
+
+        resources.load_product_index()
+
+        existing: set[tuple[int, str]] = set()
+        if skip_existing_advice:
+            existing = _existing_advice_keys(resources, as_of)
+            logger.info(
+                "%d field/disease pair(s) already have advice for %s and will be skipped",
+                len(existing),
+                as_of.isoformat(),
+            )
+
+        deadline_str = deadline_override or settings.schedule.deadline
+        deadline = _parse_deadline(deadline_str)
+        batch_start = datetime.now()
+        total_budget_s = (deadline - batch_start).total_seconds()
+        warned_80pct = False
+        warn_lock = threading.Lock()
+
+        def _run_one(job: Job) -> JobResult:
+            nonlocal warned_80pct
+
+            now = datetime.now()
+            if now >= deadline:
+                logger.error(
+                    "Deadline %s reached; skipping field=%s crop=%s disease=%s",
+                    deadline.isoformat(timespec="minutes"),
+                    job.field_id,
+                    job.crop_english,
+                    job.disease_english,
+                )
+                return JobResult(job=job, status="skipped_deadline")
+
+            if total_budget_s > 0:
+                elapsed_s = (now - batch_start).total_seconds()
+                if elapsed_s >= 0.8 * total_budget_s:
+                    with warn_lock:
+                        already_warned, warned_80pct = warned_80pct, True
+                    if not already_warned:
+                        logger.warning(
+                            "80%% of the time budget before the %s deadline has been "
+                            "used; some of the remaining %d job(s) may be skipped.",
+                            deadline_str,
+                            len(jobs),
+                        )
+
+            if (job.field_id, job.disease_english) in existing:
+                return JobResult(job=job, status="skipped_existing")
+
+            return run_job(resources, job, as_of, dry_run=dry_run)
+
+        workers = workers_override or settings.concurrency.workers
+        results: list[JobResult] = []
+        with ThreadPoolExecutor(max_workers=max(1, workers)) as pool:
+            futures = {pool.submit(_run_one, job): job for job in jobs}
+            for future in as_completed(futures):
+                job = futures[future]
+                try:
+                    result = future.result()
+                except BaseException as exc:  # noqa: BLE001 - defense in depth; run_job already isolates failures
+                    logger.exception(
+                        "Unexpected error scheduling job field=%s crop=%s disease=%s",
+                        job.field_id,
+                        job.crop_english,
+                        job.disease_english,
+                    )
+                    result = JobResult(
+                        job=job,
+                        status="failed",
+                        error_type=type(exc).__name__,
+                        error_message=str(exc),
+                    )
+
+                results.append(result)
+                if result.payload is not None:
+                    write_job_output(settings.output_dir, as_of, result.payload)
+
+        report = write_run_report(settings.output_dir, as_of, results)
+        logger.info("Batch complete: %s", {k: v for k, v in report["totals"].items() if v})
+        return 0 if report["totals"].get("failed", 0) == 0 else 1
+    finally:
+        resources.close()

pipeline/config.py

@@ -0,0 +1,234 @@
+"""Load and merge config.yaml with environment variables."""
+
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from pathlib import Path
+
+import yaml
+from dotenv import load_dotenv
+
+ROOT = Path(__file__).resolve().parent.parent
+
+
+@dataclass(frozen=True)
+class LlmSettings:
+    provider: str
+    openai_model: str
+    anthropic_model: str
+    gemini_model: str
+    max_tokens: int
+    temperature: float
+
+
+@dataclass(frozen=True)
+class SqlSettings:
+    driver: str
+    server: str
+    database: str
+    username: str
+    password: str
+
+
+@dataclass(frozen=True)
+class WeaviateSettings:
+    host: str
+    http_port: int
+    grpc_port: int
+
+
+@dataclass(frozen=True)
+class DiseasePlan:
+    """One crop-disease pair to run the batch for."""
+
+    model_name: str  # matched against AI_agrosupport_agro_models.anmod_model
+    disease: str  # canonical English disease name (vocab/diseases.yaml)
+
+
+@dataclass(frozen=True)
+class CropPlan:
+    """A crop and the diseases to check for every field growing it."""
+
+    crop: str  # canonical English crop name (vocab/crops.yaml)
+    diseases: tuple[DiseasePlan, ...]
+
+
+@dataclass(frozen=True)
+class WorklistSettings:
+    require_enabled_model: bool
+    field_allowlist: tuple[int, ...]
+
+
+@dataclass(frozen=True)
+class ConcurrencySettings:
+    workers: int
+    sql_limit: int
+    gemini_limit: int
+    gemini_requests_per_minute: int
+
+
+@dataclass(frozen=True)
+class RetrySettings:
+    attempts: int
+    initial_backoff_seconds: float
+    max_backoff_seconds: float
+
+
+@dataclass(frozen=True)
+class ScheduleSettings:
+    deadline: str  # "HH:MM", local time
+
+
+@dataclass(frozen=True)
+class Settings:
+    root: Path
+    field_id: int | None
+    disease_name: str | None
+    crops: tuple[CropPlan, ...]
+    worklist: WorklistSettings
+    concurrency: ConcurrencySettings
+    retry: RetrySettings
+    schedule: ScheduleSettings
+    crops_vocab: Path
+    diseases_vocab: Path
+    llm_query_synthesis: LlmSettings
+    llm_advice_generation: LlmSettings
+    sql: SqlSettings
+    weaviate: WeaviateSettings
+    gemini_api_key: str
+    openai_api_key: str
+    anthropic_api_key: str
+    prompts_dir: Path
+    output_dir: Path
+
+
+def _parse_llm_settings(raw: dict, defaults: dict) -> LlmSettings:
+    """Build an LlmSettings profile from a config sub-block, falling back to defaults."""
+    return LlmSettings(
+        provider=str(raw.get("provider", defaults["provider"])).strip().lower(),
+        openai_model=str(raw.get("openai_model", defaults["openai_model"])),
+        anthropic_model=str(raw.get("anthropic_model", defaults["anthropic_model"])),
+        gemini_model=str(raw.get("gemini_model", defaults["gemini_model"])),
+        max_tokens=int(raw.get("max_tokens", defaults["max_tokens"])),
+        temperature=float(raw.get("temperature", defaults["temperature"])),
+    )
+
+
+def _parse_crops(raw: list[dict] | None) -> tuple[CropPlan, ...]:
+    plans: list[CropPlan] = []
+    for entry in raw or []:
+        crop = str(entry["crop"]).strip()
+        diseases = tuple(
+            DiseasePlan(
+                model_name=str(d["model_name"]).strip(),
+                disease=str(d["disease"]).strip(),
+            )
+            for d in entry.get("diseases", [])
+        )
+        plans.append(CropPlan(crop=crop, diseases=diseases))
+    return tuple(plans)
+
+
+def _parse_worklist(raw: dict) -> WorklistSettings:
+    return WorklistSettings(
+        require_enabled_model=bool(raw.get("require_enabled_model", True)),
+        field_allowlist=tuple(int(v) for v in raw.get("field_allowlist", []) or []),
+    )
+
+
+def _parse_concurrency(raw: dict) -> ConcurrencySettings:
+    limits = raw.get("limits", {}) or {}
+    return ConcurrencySettings(
+        workers=int(raw.get("workers", 8)),
+        sql_limit=int(limits.get("sql", 6)),
+        gemini_limit=int(limits.get("gemini", 4)),
+        gemini_requests_per_minute=int(raw.get("gemini_requests_per_minute", 60)),
+    )
+
+
+def _parse_retry(raw: dict) -> RetrySettings:
+    return RetrySettings(
+        attempts=int(raw.get("attempts", 3)),
+        initial_backoff_seconds=float(raw.get("initial_backoff_seconds", 2)),
+        max_backoff_seconds=float(raw.get("max_backoff_seconds", 30)),
+    )
+
+
+def _parse_schedule(raw: dict) -> ScheduleSettings:
+    return ScheduleSettings(deadline=str(raw.get("deadline", "09:00")))
+
+
+def load_settings(
+    config_path: Path | None = None,
+    field_id_override: int | None = None,
+    disease_name_override: str | None = None,
+) -> Settings:
+    """Load YAML config and .env into a Settings object."""
+    load_dotenv(ROOT / ".env")
+    path = config_path or (ROOT / "config.yaml")
+    with path.open(encoding="utf-8") as fh:
+        raw = yaml.safe_load(fh)
+
+    llm_raw = raw.get("llm", {})
+    vocab = raw.get("vocab", {})
+
+    # field_id / disease_name only matter for `python -m pipeline one`; the
+    # batch worklist is driven entirely by the crops: block below.
+    field_id = field_id_override if field_id_override is not None else raw.get("field_id")
+    field_id = int(field_id) if field_id is not None else None
+    disease_name = disease_name_override or raw.get("disease_name")
+    disease_name = str(disease_name).strip() if disease_name is not None else None
+
+    query_synthesis_defaults = {
+        "provider": "gemini",
+        "openai_model": "gpt-4o-mini",
+        "anthropic_model": "claude-haiku-4-5",
+        "gemini_model": "gemini-2.5-flash",
+        "max_tokens": 8192,
+        "temperature": 0.0,
+    }
+    advice_generation_defaults = {
+        "provider": "gemini",
+        "openai_model": "gpt-4o",
+        "anthropic_model": "claude-opus-4-5",
+        "gemini_model": "gemini-2.5-pro",
+        "max_tokens": 65536,
+        "temperature": 0.0,
+    }
+
+    return Settings(
+        root=ROOT,
+        field_id=field_id,
+        disease_name=disease_name,
+        crops=_parse_crops(raw.get("crops")),
+        worklist=_parse_worklist(raw.get("worklist", {})),
+        concurrency=_parse_concurrency(raw.get("concurrency", {})),
+        retry=_parse_retry(raw.get("retry", {})),
+        schedule=_parse_schedule(raw.get("schedule", {})),
+        crops_vocab=ROOT / vocab.get("crops", "vocab/crops.yaml"),
+        diseases_vocab=ROOT / vocab.get("diseases", "vocab/diseases.yaml"),
+        llm_query_synthesis=_parse_llm_settings(
+            llm_raw.get("query_synthesis", {}), query_synthesis_defaults
+        ),
+        llm_advice_generation=_parse_llm_settings(
+            llm_raw.get("advice_generation", {}), advice_generation_defaults
+        ),
+        sql=SqlSettings(
+            driver=os.environ.get("SQL_DRIVER", "ODBC Driver 17 for SQL Server"),
+            server=os.environ["SQL_SERVER"],
+            database=os.environ["SQL_DATABASE"],
+            username=os.environ["SQL_USERNAME"],
+            password=os.environ["SQL_PASSWORD"],
+        ),
+        weaviate=WeaviateSettings(
+            host=os.environ.get("WEAVIATE_HOST", "localhost"),
+            http_port=int(os.environ.get("WEAVIATE_HTTP_PORT", "8080")),
+            grpc_port=int(os.environ.get("WEAVIATE_GRPC_PORT", "50051")),
+        ),
+        gemini_api_key=os.environ.get("GEMINI_API_KEY", ""),
+        openai_api_key=os.environ.get("OPENAI_API_KEY", ""),
+        anthropic_api_key=os.environ.get("ANTHROPIC_API_KEY", ""),
+        prompts_dir=ROOT / "prompts",
+        output_dir=ROOT / "output",
+    )

pipeline/db.py

@@ -0,0 +1,86 @@
+"""SQL Server connection helpers via pyodbc."""
+
+from __future__ import annotations
+
+from contextlib import contextmanager
+from typing import Any, Iterator
+
+import pyodbc
+
+from pipeline.config import SqlSettings
+
+
+def _connection_string(settings: SqlSettings) -> str:
+    return (
+        f"DRIVER={{{settings.driver}}};"
+        f"SERVER={settings.server};"
+        f"DATABASE={settings.database};"
+        f"UID={settings.username};"
+        f"PWD={settings.password};"
+        "TrustServerCertificate=yes;"
+    )
+
+
+def connect_raw(settings: SqlSettings) -> pyodbc.Connection:
+    """
+    Open a new pyodbc connection without any context-manager lifecycle.
+
+    Used by the batch orchestrator to hand each worker thread its own
+    long-lived connection (pyodbc connections must not be shared across
+    threads); the caller is responsible for closing it.
+    """
+    return pyodbc.connect(_connection_string(settings), autocommit=True)
+
+
+@contextmanager
+def connect(settings: SqlSettings) -> Iterator[pyodbc.Connection]:
+    """Yield an open pyodbc connection that is closed on exit."""
+    conn = connect_raw(settings)
+    try:
+        yield conn
+    finally:
+        conn.close()
+
+
+def fetch_one(conn: pyodbc.Connection, sql: str, params: tuple[Any, ...] = ()) -> pyodbc.Row | None:
+    cursor = conn.cursor()
+    cursor.execute(sql, params)
+    return cursor.fetchone()
+
+
+def fetch_all(conn: pyodbc.Connection, sql: str, params: tuple[Any, ...] = ()) -> list[pyodbc.Row]:
+    cursor = conn.cursor()
+    cursor.execute(sql, params)
+    return list(cursor.fetchall())
+
+
+def execute(conn: pyodbc.Connection, sql: str, params: tuple[Any, ...] = ()) -> int:
+    """Run a write statement and return the affected row count (autocommit is on)."""
+    cursor = conn.cursor()
+    cursor.execute(sql, params)
+    return cursor.rowcount
+
+
+@contextmanager
+def transaction(conn: pyodbc.Connection) -> Iterator[None]:
+    """
+    Run a block of statements atomically on a connection that normally runs
+    with autocommit on.
+
+    Used by `insert_advice`'s DELETE-then-INSERT pair: under concurrent
+    workers, autocommit would let a crash between the two statements leave a
+    day's advisory missing. This temporarily disables autocommit, commits on
+    success, rolls back on any exception, and always restores the previous
+    autocommit mode afterwards so the connection is safe to reuse for the
+    next job.
+    """
+    previous_autocommit = conn.autocommit
+    conn.autocommit = False
+    try:
+        yield
+        conn.commit()
+    except Exception:
+        conn.rollback()
+        raise
+    finally:
+        conn.autocommit = previous_autocommit

pipeline/errors.py

@@ -0,0 +1,9 @@
+"""Pipeline-specific exceptions."""
+
+
+class PipelineError(Exception):
+    """Base error for the agronomic pipeline."""
+
+
+class NoModelConfiguredError(PipelineError):
+    """Raised when no disease forecast model exists for the target disease."""

pipeline/job.py

@@ -0,0 +1,302 @@
+"""Run the full single-field pipeline for one (field, crop, disease) job.
+
+This is the per-job unit of work the batch orchestrator (`pipeline.batch`)
+dispatches to its thread pool. It is the same sequence the original
+single-field CLI ran (resolve disease model -> weather/phenology/treatments ->
+query synthesis -> product prefilter -> vector search -> advice generation ->
+persistence), restructured to take a `worklist.Job` (crop, organic flag,
+station, and anmod_id already resolved by the worklist join) plus shared
+`Resources`, instead of reading `field_id` / `disease_name` from config and
+opening its own SQL/Weaviate connections.
+
+`run_job` never raises: any failure — from either this module or anything it
+calls — comes back as a `failed` JobResult, so one field's bad data or a
+transient API error cannot stop the batch from processing every other field.
+"""
+
+from __future__ import annotations
+
+import logging
+import time
+from dataclasses import dataclass
+from datetime import date
+from decimal import Decimal
+from typing import Any
+
+from pipeline.errors import PipelineError
+from pipeline.prompts import pair_slug
+from pipeline.resources import Resources
+from pipeline.retry import call_with_sql_retry
+from pipeline.stages.advice import generate_advice, insert_advice
+from pipeline.stages.advice_context import (
+    collapse_product_json,
+    enrich_advice_json,
+    load_last_advice,
+)
+from pipeline.stages.assemble import (
+    assemble_json_for_advice_generation,
+    build_json_for_query_synthesis,
+)
+from pipeline.stages.disease import nearest_fase, resolve_disease_forecast_by_anmod
+from pipeline.stages.llm import synthesize_queries
+from pipeline.stages.phenology import load_phenology
+from pipeline.stages.products import prefilter_products
+from pipeline.stages.treatments import load_applied_treatments
+from pipeline.stages.vector import RecommendedProduct
+from pipeline.stages.weather import load_weather
+from pipeline.worklist import Job
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class JobResult:
+    """Outcome of one job. Constructed either by `run_job` or, for jobs the
+    batch orchestrator decides not to run at all, directly by `pipeline.batch`
+    (`skipped_existing`, `skipped_deadline`)."""
+
+    job: Job
+    status: str  # "ok" | "no_risk" | "dry_run" | "failed" | "skipped_existing" | "skipped_deadline"
+    duration_s: float = 0.0
+    error_type: str | None = None
+    error_message: str | None = None
+    recommended_count: int = 0
+    allowed_products_empty: bool = False
+    message: str | None = None
+    payload: dict[str, Any] | None = None
+
+
+def json_default(obj: Any) -> Any:
+    """`json.dump(..., default=...)` hook for the `date`/`Decimal` values in a job payload."""
+    if isinstance(obj, date):
+        return obj.isoformat()
+    if isinstance(obj, Decimal):
+        return float(obj)
+    raise TypeError(f"Object of type {type(obj)!r} is not JSON serializable")
+
+
+def output_stem(job: Job, disease_english: str) -> str:
+    """File name stem (no extension) for this job's per-field output JSON."""
+    return f"field{job.field_id}__{pair_slug(job.crop_english, disease_english)}"
+
+
+def run_job(resources: Resources, job: Job, as_of: date, dry_run: bool = False) -> JobResult:
+    """Run the full pipeline for one job."""
+    started = time.monotonic()
+    try:
+        payload = _run_job_inner(resources, job, as_of, dry_run)
+    except BaseException as exc:  # noqa: BLE001 - fault isolation is the whole point of this wrapper
+        logger.exception(
+            "Job failed: field=%s crop=%s disease=%s",
+            job.field_id,
+            job.crop_english,
+            job.disease_english,
+        )
+        return JobResult(
+            job=job,
+            status="failed",
+            duration_s=time.monotonic() - started,
+            error_type=type(exc).__name__,
+            error_message=str(exc),
+        )
+
+    status = payload["status"]
+    allowed_products_empty = False
+    if status in ("ok", "no_risk"):
+        allowed = (payload.get("json_for_advice_generation") or {}).get("allowed_products")
+        allowed_products_empty = not allowed
+
+    return JobResult(
+        job=job,
+        status=status,
+        duration_s=time.monotonic() - started,
+        recommended_count=len(payload.get("recommended_products") or []),
+        allowed_products_empty=allowed_products_empty,
+        message=payload.get("message"),
+        payload=payload,
+    )
+
+
+def _run_job_inner(resources: Resources, job: Job, as_of: date, dry_run: bool) -> dict[str, Any]:
+    settings = resources.settings
+    conn = resources.sql_connection()
+
+    with resources.sql_sem:
+        disease = resolve_disease_forecast_by_anmod(
+            conn,
+            anmod_id=job.anmod_id,
+            disease_name=job.model_name,
+            disease_english=job.disease_english,
+            as_of=as_of,
+            fallback_station=job.station,
+        )
+
+        if disease.station is None:
+            raise PipelineError(
+                f"Field {job.field_id}: unable to determine weather station "
+                "(model_station / cmplay_station both missing)."
+            )
+
+        weather = load_weather(conn, disease.station, as_of)
+        phenology = load_phenology(conn, job.field_id, as_of)
+        applied_treatments = load_applied_treatments(conn, job.field_id, as_of)
+
+    logger.info(
+        "Field %s (%s/%s): station=%s has_risk=%s first_future=%s",
+        job.field_id,
+        job.crop_english,
+        job.disease_english,
+        disease.station,
+        disease.has_risk,
+        disease.first_future_disease_date,
+    )
+
+    json_for_advice_generation = assemble_json_for_advice_generation(
+        as_of,
+        weather,
+        phenology,
+        disease.forecasts_by_day,
+        applied_treatments,
+    )
+    json_for_query_synthesis = build_json_for_query_synthesis(
+        as_of,
+        job.crop_english,
+        disease.disease_english,
+        weather,
+        phenology,
+        disease.first_future_disease_date,
+    )
+
+    payload: dict[str, Any] = {
+        "run": {
+            "as_of": as_of.isoformat(),
+            "field_id": job.field_id,
+            "crop": job.crop_english,
+            "crop_italian": job.crop_italian,
+            "disease": disease.disease_english,
+            "disease_raw": disease.disease_raw,
+            "organic": job.organic,
+            "station": disease.station,
+            "anmod_id": disease.anmod_id,
+            "has_risk": disease.has_risk,
+            "first_future_disease_date": (
+                disease.first_future_disease_date.isoformat()
+                if disease.first_future_disease_date
+                else None
+            ),
+            "llm_provider": settings.llm_query_synthesis.provider,
+        },
+        "json_for_advice_generation": json_for_advice_generation,
+        "json_for_query_synthesis": json_for_query_synthesis,
+        "generated_queries": [],
+        "candidate_count": 0,
+        "recommended_products": [],
+        "last_advice": None,
+        "advice": None,
+        "sent_information": None,
+        "advice_inserted": False,
+        "status": "ok",
+        "message": None,
+        "output_stem": output_stem(job, disease.disease_english),
+    }
+
+    if dry_run:
+        payload["status"] = "dry_run"
+        payload["message"] = (
+            "Dry run: skipped LLM, vector search, advice generation, and DB write."
+        )
+        return payload
+
+    recommended: list[RecommendedProduct] = []
+    if disease.has_risk:
+        query_system_prompt = resources.prompts.query_prompt(job.crop_english, disease.disease_english)
+        queries = synthesize_queries(
+            settings,
+            query_system_prompt,
+            json_for_query_synthesis,
+            llm_call=resources.call_llm,
+        )
+        payload["generated_queries"] = queries
+
+        fase = nearest_fase(disease.forecasts_by_day, as_of)
+        candidates = prefilter_products(
+            resources.product_index,
+            crop_english=job.crop_english,
+            disease_english=disease.disease_english,
+            organic=job.organic,
+            fase=fase,
+        )
+        payload["candidate_count"] = len(candidates)
+
+        recommended = resources.search_products(queries, candidates)
+        payload["recommended_products"] = [
+            {
+                "product_name": item.product_name,
+                "registration_number": item.registration_number,
+                "distance": item.distance,
+                "query_index": item.query_index,
+                "product_id": item.product_id,
+            }
+            for item in recommended
+        ]
+        logger.info(
+            "Field %s: recommended products: %s",
+            job.field_id,
+            [p["product_name"] for p in payload["recommended_products"]],
+        )
+    else:
+        payload["status"] = "no_risk"
+        logger.warning(
+            "Field %s: neither FASE2 nor FASE5 in the 11-day window; product "
+            "search skipped, advice generated without allowed products.",
+            job.field_id,
+        )
+
+    builder = resources.product_json_builder(job.crop_english, disease.disease_english)
+    with resources.sql_sem:
+        last_summary, last_treatments = load_last_advice(
+            conn, job.field_id, disease.disease_english, as_of
+        )
+        enrich_advice_json(
+            json_for_advice_generation,
+            conn=conn,
+            builder=builder,
+            recommended=recommended,
+            last_advice_summary=last_summary,
+            last_advice_treatments=last_treatments,
+        )
+    payload["last_advice"] = json_for_advice_generation["last_advice"]
+
+    advice_system_prompt, advice_user_prompt = resources.prompts.advice_prompts(
+        job.crop_english, disease.disease_english
+    )
+    result = generate_advice(
+        settings,
+        advice_system_prompt,
+        advice_user_prompt,
+        json_for_advice_generation,
+        llm_call=resources.call_llm,
+    )
+    sent_information = collapse_product_json(json_for_advice_generation)
+
+    def _insert() -> None:
+        with resources.sql_sem:
+            insert_advice(
+                conn,
+                run_date=as_of,
+                field_id=job.field_id,
+                disease_english=disease.disease_english,
+                sent_information=sent_information,
+                result=result,
+            )
+
+    call_with_sql_retry(settings.retry, _insert)
+
+    payload["advice"] = result.as_dict()
+    payload["sent_information"] = sent_information
+    payload["advice_inserted"] = True
+    payload["message"] = (
+        f"Returned {len(recommended)} recommended product(s); "
+        f"advice stored with apply_treatment={int(result.apply_treatment)}."
+    )
+    return payload

pipeline/jsonutils.py

@@ -0,0 +1,29 @@
+"""Helpers for the JSON-encoded list columns stored as text in SQL Server."""
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+
+def parse_json_list(raw: Any) -> list[str]:
+    """Decode a JSON array column into a list of strings; unparseable input yields []."""
+    if raw is None:
+        return []
+    if isinstance(raw, list):
+        return [str(item) for item in raw]
+    text = str(raw).strip()
+    if not text:
+        return []
+    try:
+        parsed = json.loads(text)
+        if isinstance(parsed, list):
+            return [str(item) for item in parsed]
+    except json.JSONDecodeError:
+        pass
+    return []
+
+
+def normalize_name(product_name: str) -> str:
+    """Key used for case- and whitespace-insensitive product name comparison."""
+    return product_name.strip().casefold()

pipeline/output.py

@@ -0,0 +1,104 @@
+"""Per-run output layout: one JSON file per job plus a daily run report.
+
+The single-field CLI wrote `output/<date>_field<id>.json`, which collides as
+soon as one field has two diseases. The batch instead writes one file per
+(field, disease) pair under a per-day directory, plus a `_run_report.json`
+that is the actual operational artifact: it says whether the morning's run
+succeeded, and if not, exactly which jobs failed and why.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from datetime import date
+from pathlib import Path
+from typing import Any
+
+from pipeline.job import JobResult, json_default
+
+logger = logging.getLogger(__name__)
+
+# Keys always present in the report's totals, even at zero, so a dashboard or
+# script reading the report does not need to guess which statuses exist.
+_STATUSES = (
+    "ok",
+    "no_risk",
+    "dry_run",
+    "failed",
+    "skipped_existing",
+    "skipped_deadline",
+)
+
+
+def run_dir(output_dir: Path, as_of: date) -> Path:
+    return output_dir / as_of.isoformat()
+
+
+def write_job_output(output_dir: Path, as_of: date, payload: dict[str, Any]) -> Path:
+    """Write one job's full payload to output/<as_of>/<output_stem>.json."""
+    target_dir = run_dir(output_dir, as_of)
+    target_dir.mkdir(parents=True, exist_ok=True)
+    path = target_dir / f"{payload['output_stem']}.json"
+    with path.open("w", encoding="utf-8") as fh:
+        json.dump(payload, fh, ensure_ascii=False, indent=2, default=json_default)
+    return path
+
+
+def _job_report_entry(result: JobResult) -> dict[str, Any]:
+    job = result.job
+    return {
+        "field_id": job.field_id,
+        "crop": job.crop_english,
+        "disease": job.disease_english,
+        "organic": job.organic,
+        "station": job.station,
+        "status": result.status,
+        "duration_s": round(result.duration_s, 2),
+        "recommended_count": result.recommended_count,
+        "allowed_products_empty": result.allowed_products_empty,
+        "message": result.message,
+        "error_type": result.error_type,
+        "error_message": result.error_message,
+    }
+
+
+def write_run_report(
+    output_dir: Path,
+    as_of: date,
+    results: list[JobResult],
+) -> dict[str, Any]:
+    """
+    Write output/<as_of>/_run_report.json: totals by status plus per-job
+    diagnostics.
+
+    This is the operational artifact for the daily run: it names exactly
+    which jobs succeeded, which were skipped and why, and which failed and
+    with what error, so the exit code alone never has to carry that
+    information. `ok_with_empty_allowed_products` is tracked separately from
+    a bare count of `ok` jobs because, once the `products` table has organic
+    entries for every crop, a sudden jump in that number is the fastest
+    signal that the prefilter or the Weaviate allowlist filter has regressed
+    (see pipeline/stages/vector.py).
+    """
+    totals = {status: 0 for status in _STATUSES}
+    for result in results:
+        totals[result.status] = totals.get(result.status, 0) + 1
+
+    empty_allowed_products = sum(1 for r in results if r.allowed_products_empty)
+
+    report: dict[str, Any] = {
+        "as_of": as_of.isoformat(),
+        "job_count": len(results),
+        "totals": totals,
+        "ok_with_empty_allowed_products": empty_allowed_products,
+        "jobs": [_job_report_entry(r) for r in results],
+    }
+
+    target_dir = run_dir(output_dir, as_of)
+    target_dir.mkdir(parents=True, exist_ok=True)
+    path = target_dir / "_run_report.json"
+    with path.open("w", encoding="utf-8") as fh:
+        json.dump(report, fh, ensure_ascii=False, indent=2)
+    logger.info("Wrote run report -> %s", path)
+    return report

pipeline/prompts.py

@@ -0,0 +1,125 @@
+"""Per-crop-disease prompt registry.
+
+Every crop grows a specific set of diseases, and each crop-disease pair needs a
+tailored advice prompt (its own phytopathological model semantics, phenology
+notes, etc.). Prompts live on disk under `prompts/` and are resolved by a
+`(crop, disease)` slug, with a shared `_default` prompt for pairs that have not
+been given a dedicated one yet:
+
+    prompts/_output_format.md                          shared advice JSON contract
+    prompts/query_synthesis/_default.md                 generic query-synthesis system prompt
+    prompts/query_synthesis/<crop>__<disease>.md         optional per-pair override
+    prompts/advice/_default/{system.md,user.md}          fallback advice prompts
+    prompts/advice/<crop>__<disease>/{system.md,user.md} per-pair advice prompts
+
+All prompts are read once and cached in memory, since the batch resolves them
+from multiple worker threads.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+import threading
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Iterable
+
+from pipeline.errors import PipelineError
+
+logger = logging.getLogger(__name__)
+
+_DEFAULT_SLUG = "_default"
+
+
+def slugify(text: str) -> str:
+    """Normalise a crop or disease name into a filesystem-safe slug."""
+    slug = re.sub(r"[^a-z0-9]+", "_", text.strip().lower())
+    return slug.strip("_")
+
+
+def pair_slug(crop: str, disease: str) -> str:
+    """The directory / filename stem identifying a crop-disease pair."""
+    return f"{slugify(crop)}__{slugify(disease)}"
+
+
+@dataclass(frozen=True)
+class _Resolved:
+    text: str
+    is_default: bool
+
+
+class PromptRegistry:
+    """Loads and caches advice / query-synthesis prompts from `prompts_dir`."""
+
+    def __init__(self, prompts_dir: Path) -> None:
+        self._dir = prompts_dir
+        self._output_format = self._read(prompts_dir / "_output_format.md")
+        self._lock = threading.Lock()
+        self._advice_cache: dict[tuple[str, str], tuple[_Resolved, _Resolved]] = {}
+        self._query_cache: dict[tuple[str, str], str] = {}
+
+    @staticmethod
+    def _read(path: Path) -> str:
+        if not path.is_file():
+            raise PipelineError(f"Prompt file not found: {path}")
+        return path.read_text(encoding="utf-8").strip()
+
+    def _resolve_advice_file(self, slug: str, filename: str) -> _Resolved:
+        advice_dir = self._dir / "advice"
+        specific = advice_dir / slug / filename
+        if specific.is_file():
+            return _Resolved(text=self._read(specific), is_default=False)
+        default = advice_dir / _DEFAULT_SLUG / filename
+        return _Resolved(text=self._read(default), is_default=True)
+
+    def _get_advice(self, crop: str, disease: str) -> tuple[_Resolved, _Resolved]:
+        slug = pair_slug(crop, disease)
+        with self._lock:
+            cached = self._advice_cache.get(slug)
+            if cached is not None:
+                return cached
+            resolved = (
+                self._resolve_advice_file(slug, "system.md"),
+                self._resolve_advice_file(slug, "user.md"),
+            )
+            self._advice_cache[slug] = resolved
+            return resolved
+
+    def advice_prompts(self, crop: str, disease: str) -> tuple[str, str]:
+        """Return (system_prompt, user_prompt) for a crop-disease pair.
+
+        The system prompt is the crop-disease-specific (or default) prompt
+        followed by the shared output-format contract, so every advice call
+        gets the same strict JSON schema regardless of which prompt matched.
+        """
+        system, user = self._get_advice(crop, disease)
+        full_system = f"{system.text}\n\n---\n\n{self._output_format}"
+        return full_system, user.text
+
+    def query_prompt(self, crop: str, disease: str) -> str:
+        """Return the query-synthesis system prompt for a crop-disease pair."""
+        slug = pair_slug(crop, disease)
+        with self._lock:
+            cached = self._query_cache.get(slug)
+            if cached is not None:
+                return cached
+            qs_dir = self._dir / "query_synthesis"
+            specific = qs_dir / f"{slug}.md"
+            text = self._read(specific) if specific.is_file() else self._read(qs_dir / f"{_DEFAULT_SLUG}.md")
+            self._query_cache[slug] = text
+            return text
+
+    def validate(self, pairs: Iterable[tuple[str, str]]) -> list[str]:
+        """
+        Return "crop / disease" strings for pairs without a dedicated advice prompt.
+
+        Call before starting the worker pool so missing prompts are reported
+        up front instead of discovered mid-batch.
+        """
+        fallbacks: list[str] = []
+        for crop, disease in pairs:
+            system, user = self._get_advice(crop, disease)
+            if system.is_default or user.is_default:
+                fallbacks.append(f"{crop} / {disease}")
+        return fallbacks

pipeline/resources.py

@@ -0,0 +1,200 @@
+"""Resources shared across every job in a batch run.
+
+A single-field CLI run opened one SQL connection, one Weaviate client, and
+made its two LLM calls without needing to coordinate with anyone else. The
+batch runs many jobs concurrently across a thread pool, so several things
+that used to be "create it, use it, close it" per field now need to be
+long-lived, shared, and safe to touch from multiple threads at once:
+
+- pyodbc connections must not be shared across threads, so each worker thread
+  gets its own, created lazily and reused for every job it picks up.
+- The Weaviate client is documented as safe for concurrent query use, so one
+  instance is opened for the whole batch instead of one per field.
+- `near_text` search embeds through the same Gemini API key as the two LLM
+  calls (`ProductProfile` is vectorised with text2vec-palm /
+  gemini-embedding-001), so a single semaphore + rate limiter covers all three
+  Gemini-backed calls per job instead of treating "LLM" and "Weaviate" as
+  independent budgets.
+- The `products` table and the `ProductJsonBuilder` per-product cache are
+  loaded/shared once instead of once per field.
+"""
+
+from __future__ import annotations
+
+import logging
+import threading
+import time
+from typing import Any
+
+from pipeline.config import LlmSettings, Settings
+from pipeline.db import connect_raw
+from pipeline.prompts import PromptRegistry
+from pipeline.retry import call_with_llm_retry, call_with_weaviate_retry
+from pipeline.stages.llm import call_llm as _raw_call_llm
+from pipeline.stages.product_json import ProductJsonBuilder
+from pipeline.stages.products import ProductCandidate, ProductIndex
+from pipeline.stages.vector import RecommendedProduct, connect_client
+from pipeline.stages.vector import search_products as _raw_search_products
+
+logger = logging.getLogger(__name__)
+
+
+class RateLimiter:
+    """
+    Thread-safe pacing limiter: spreads calls out to at most N per minute.
+
+    This paces call *starts* evenly (a leaky bucket, not a bursty token
+    bucket with saved-up credit) — the simplest thing that reliably keeps a
+    pool of worker threads under an external rate limit, which matters more
+    here than allowing occasional bursts.
+    """
+
+    def __init__(self, requests_per_minute: int) -> None:
+        self._interval = 60.0 / requests_per_minute if requests_per_minute > 0 else 0.0
+        self._lock = threading.Lock()
+        self._next_allowed = 0.0
+
+    def acquire(self) -> None:
+        if self._interval <= 0:
+            return
+        with self._lock:
+            now = time.monotonic()
+            wait = self._next_allowed - now
+            if wait > 0:
+                time.sleep(wait)
+                now = time.monotonic()
+            self._next_allowed = max(now, self._next_allowed) + self._interval
+
+
+class Resources:
+    """Shared, thread-safe state for one batch run. Create once, close once."""
+
+    def __init__(self, settings: Settings, prompts: PromptRegistry) -> None:
+        self.settings = settings
+        self.prompts = prompts
+
+        self._local = threading.local()
+        self._connections: list[Any] = []
+        self._connections_lock = threading.Lock()
+        self.sql_sem = threading.Semaphore(settings.concurrency.sql_limit)
+
+        self.gemini_sem = threading.Semaphore(settings.concurrency.gemini_limit)
+        self.gemini_rate_limiter = RateLimiter(settings.concurrency.gemini_requests_per_minute)
+
+        self._weaviate_client: Any = None
+        self._weaviate_lock = threading.Lock()
+
+        self.product_index: ProductIndex | None = None
+
+        self._builder_cache: dict[tuple[str, str], ProductJsonBuilder] = {}
+        self._builder_lock = threading.Lock()
+
+    # -- SQL -----------------------------------------------------------------
+
+    def sql_connection(self) -> Any:
+        """
+        Return this thread's SQL connection, opening one on first use.
+
+        Not gated by `sql_sem`: with one connection per worker thread kept
+        open for the batch's whole lifetime, gating *creation* would only
+        throttle how fast the pool warms up, and gating it for the
+        connection's entire lifetime would deadlock as soon as `sql_limit` is
+        set below `workers` (the remaining threads would block forever on a
+        permit nothing ever releases). `sql_sem` instead bounds how many jobs
+        may run their SQL-heavy phase at the same instant — see
+        `pipeline.job.run_job`, which acquires it around each phase, not
+        around the connection.
+        """
+        conn = getattr(self._local, "conn", None)
+        if conn is None:
+            conn = connect_raw(self.settings.sql)
+            self._local.conn = conn
+            with self._connections_lock:
+                self._connections.append(conn)
+        return conn
+
+    def load_product_index(self) -> ProductIndex:
+        """Load the whole `products` table once; call before starting the pool."""
+        self.product_index = ProductIndex.load(self.sql_connection())
+        return self.product_index
+
+    def product_json_builder(self, crop_english: str, disease_english: str) -> ProductJsonBuilder:
+        """
+        One `ProductJsonBuilder` per (crop, disease), shared by every field
+        growing that crop with that disease across all worker threads.
+        """
+        if self.product_index is None:
+            raise RuntimeError("load_product_index() must run before product_json_builder().")
+        key = (crop_english, disease_english)
+        with self._builder_lock:
+            builder = self._builder_cache.get(key)
+            if builder is None:
+                builder = ProductJsonBuilder(self.product_index, crop_english, disease_english)
+                self._builder_cache[key] = builder
+            return builder
+
+    # -- Weaviate / Gemini -----------------------------------------------------
+
+    def _weaviate(self) -> Any:
+        if self._weaviate_client is None:
+            with self._weaviate_lock:
+                if self._weaviate_client is None:
+                    self._weaviate_client = connect_client(self.settings)
+        return self._weaviate_client
+
+    def call_llm(
+        self,
+        settings: Settings,
+        llm_cfg: LlmSettings,
+        system_prompt: str,
+        user_content: str,
+        json_output: bool = False,
+    ) -> str:
+        """
+        Rate-limited, retried, concurrency-bounded drop-in for
+        `pipeline.stages.llm.call_llm`.
+
+        Signature-compatible so it can be passed as `synthesize_queries(...,
+        llm_call=resources.call_llm)` / `generate_advice(...,
+        llm_call=resources.call_llm)`.
+        """
+
+        def _attempt() -> str:
+            with self.gemini_sem:
+                self.gemini_rate_limiter.acquire()
+                return _raw_call_llm(settings, llm_cfg, system_prompt, user_content, json_output)
+
+        return call_with_llm_retry(self.settings.retry, _attempt)
+
+    def search_products(
+        self,
+        queries: list[str],
+        candidates: list[ProductCandidate],
+    ) -> list[RecommendedProduct]:
+        """Rate-limited, retried wrapper around `pipeline.stages.vector.search_products`."""
+
+        def _attempt() -> list[RecommendedProduct]:
+            with self.gemini_sem:
+                self.gemini_rate_limiter.acquire()
+                return _raw_search_products(self._weaviate(), queries, candidates)
+
+        return call_with_weaviate_retry(self.settings.retry, _attempt)
+
+    # -- Lifecycle -------------------------------------------------------------
+
+    def close(self) -> None:
+        """Close every pooled SQL connection and the shared Weaviate client."""
+        with self._connections_lock:
+            for conn in self._connections:
+                try:
+                    conn.close()
+                except Exception:
+                    logger.warning("Error closing a pooled SQL connection.", exc_info=True)
+            self._connections.clear()
+
+        if self._weaviate_client is not None:
+            try:
+                self._weaviate_client.close()
+            except Exception:
+                logger.warning("Error closing the Weaviate client.", exc_info=True)
+            self._weaviate_client = None

pipeline/retry.py

@@ -0,0 +1,110 @@
+"""Retry policies for the three external boundaries the batch depends on:
+LLM providers, Weaviate, and SQL Server. `tenacity` was already a pinned
+dependency (see requirements.txt) but unused until now — the original
+single-field CLI ran once a day and a failure just meant re-running by hand.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Callable, TypeVar
+
+import pyodbc
+from tenacity import (
+    RetryCallState,
+    Retrying,
+    retry_if_exception,
+    stop_after_attempt,
+    wait_exponential_jitter,
+)
+
+from pipeline.config import RetrySettings
+from pipeline.errors import NoModelConfiguredError, PipelineError
+
+logger = logging.getLogger(__name__)
+
+T = TypeVar("T")
+
+# SQL Server deadlock victim / serialization failure - safe to retry, the
+# transaction was rolled back and never committed.
+_DEADLOCK_SQLSTATES = {"40001"}
+# Link failure / connection timeout - safe to retry a fresh statement.
+_CONNECTION_SQLSTATES = {"08S01", "08001", "HYT00", "HYT01"}
+
+
+def _log_retry(retry_state: RetryCallState) -> None:
+    exc = retry_state.outcome.exception() if retry_state.outcome else None
+    fn_name = getattr(retry_state.fn, "__name__", "call")
+    logger.warning(
+        "Retrying %s after attempt %d (%s: %s)",
+        fn_name,
+        retry_state.attempt_number,
+        type(exc).__name__ if exc else "?",
+        exc,
+    )
+
+
+def is_llm_retryable(exc: BaseException) -> bool:
+    """
+    Transient LLM failures worth a retry: HTTP 429 / 5xx, timeouts, and
+    transport errors from any of the three SDKs, plus a `PipelineError`
+    raised because the model returned empty or unparseable output (a re-roll
+    often produces valid output on the next attempt).
+
+    Never retried: `NoModelConfiguredError` and other pipeline errors that
+    are not about LLM output — those describe a configuration or data
+    problem that will fail identically on every attempt.
+    """
+    if isinstance(exc, NoModelConfiguredError):
+        return False
+    if isinstance(exc, PipelineError):
+        return True
+
+    status_code = getattr(exc, "status_code", None)
+    if isinstance(status_code, int) and (status_code == 429 or status_code >= 500):
+        return True
+
+    name = type(exc).__name__
+    return any(
+        token in name
+        for token in ("RateLimit", "APIConnection", "APITimeout", "Timeout", "ServiceUnavailable")
+    )
+
+
+def is_weaviate_retryable(exc: BaseException) -> bool:
+    """Connection/timeout/availability failures from the weaviate-client SDK."""
+    name = type(exc).__name__
+    return any(token in name for token in ("Timeout", "Connection", "Unavailable", "Deadline"))
+
+
+def is_sql_retryable(exc: BaseException) -> bool:
+    """Deadlocks and transient connection failures only; anything else is a real bug."""
+    if not isinstance(exc, pyodbc.Error):
+        return False
+    sqlstate = exc.args[0] if exc.args else ""
+    return sqlstate in _DEADLOCK_SQLSTATES or sqlstate in _CONNECTION_SQLSTATES
+
+
+def _make_retrying(retry_settings: RetrySettings, predicate: Callable[[BaseException], bool]) -> Retrying:
+    return Retrying(
+        stop=stop_after_attempt(max(1, retry_settings.attempts)),
+        wait=wait_exponential_jitter(
+            initial=retry_settings.initial_backoff_seconds,
+            max=retry_settings.max_backoff_seconds,
+        ),
+        retry=retry_if_exception(predicate),
+        before_sleep=_log_retry,
+        reraise=True,
+    )
+
+
+def call_with_llm_retry(retry_settings: RetrySettings, fn: Callable[..., T], *args, **kwargs) -> T:
+    return _make_retrying(retry_settings, is_llm_retryable)(fn, *args, **kwargs)
+
+
+def call_with_weaviate_retry(retry_settings: RetrySettings, fn: Callable[..., T], *args, **kwargs) -> T:
+    return _make_retrying(retry_settings, is_weaviate_retryable)(fn, *args, **kwargs)
+
+
+def call_with_sql_retry(retry_settings: RetrySettings, fn: Callable[..., T], *args, **kwargs) -> T:
+    return _make_retrying(retry_settings, is_sql_retryable)(fn, *args, **kwargs)

pipeline/stages/__init__.py

@@ -0,0 +1 @@
+"""Stage modules for the agronomic pipeline."""

pipeline/stages/advice.py

@@ -0,0 +1,274 @@
+"""Part_Two, step 3: generate the farmer advisory and persist it to the advice table."""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from dataclasses import dataclass
+from datetime import date, datetime
+from typing import Any, Callable
+
+import pyodbc
+
+from pipeline.config import Settings
+from pipeline.db import execute, transaction
+from pipeline.errors import PipelineError
+from pipeline.stages.llm import call_llm
+
+logger = logging.getLogger(__name__)
+
+# advice.advice_summary is nvarchar(1050); the prompt asks for at most 1010 chars.
+SUMMARY_MAX_CHARS = 1010
+_TREATMENTS_MAX_CHARS = 200
+_DOSAGE_MAX_CHARS = 100
+
+_DELETE_ADVICE_SQL = """
+DELETE FROM advice
+WHERE [date] = ?
+  AND field_id = ?
+  AND disease = ?
+"""
+
+_INSERT_ADVICE_SQL = """
+INSERT INTO advice (
+    [date],
+    field_id,
+    disease,
+    sent_information,
+    full_advice,
+    advice_summary,
+    apply_treatment,
+    treatments,
+    dosage,
+    apply_date
+) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+"""
+
+
+@dataclass(frozen=True)
+class AdviceResult:
+    full_advice: str
+    advice_summary: str
+    apply_treatment: bool
+    treatments: list[str]
+    dosage: list[str]
+    apply_date: date | None
+
+    def as_dict(self) -> dict[str, Any]:
+        return {
+            "full_advice": self.full_advice,
+            "advice_summary": self.advice_summary,
+            "apply_treatment": int(self.apply_treatment),
+            "treatments": self.treatments,
+            "dosage": self.dosage,
+            "apply_date": self.apply_date.isoformat() if self.apply_date else None,
+        }
+
+
+def generate_advice(
+    settings: Settings,
+    system_prompt: str,
+    user_prompt: str,
+    advice_json: dict[str, Any],
+    llm_call: Callable[..., str] = None,
+) -> AdviceResult:
+    """
+    Send json_for_advice_generation to the advice LLM and parse the structured reply.
+
+    `llm_call` defaults to the plain, unguarded `call_llm` (fine for the
+    single-field `one` CLI path). The batch orchestrator passes
+    `resources.call_llm` instead, which adds the shared Gemini semaphore,
+    rate limiting, and retry policy around the same call signature.
+    """
+    llm_call = llm_call or call_llm
+    user_content = (
+        f"{user_prompt}\n\n{json.dumps(advice_json, ensure_ascii=False, indent=2, default=_json_default)}"
+    )
+
+    raw = llm_call(
+        settings,
+        settings.llm_advice_generation,
+        system_prompt,
+        user_content,
+        json_output=True,
+    )
+    return parse_advice(raw)
+
+
+def parse_advice(raw: str) -> AdviceResult:
+    """Parse and validate the advice JSON returned by the LLM."""
+    payload = _load_json_object(raw)
+
+    full_advice = _require_text(payload, "full_advice")
+    summary = _require_text(payload, "advice_summary")
+    if len(summary) > SUMMARY_MAX_CHARS:
+        logger.warning(
+            "advice_summary is %d characters; truncating to %d.",
+            len(summary),
+            SUMMARY_MAX_CHARS,
+        )
+        summary = summary[:SUMMARY_MAX_CHARS]
+
+    treatments = _string_list(payload.get("treatments"))
+    dosage = _string_list(payload.get("dosage"))
+    apply_treatment = _to_bool(payload.get("apply_treatment"))
+    apply_date = _to_date(payload.get("apply_date"))
+
+    if apply_treatment and not treatments:
+        logger.warning("apply_treatment is set but treatments is empty.")
+    if not apply_treatment and treatments:
+        logger.warning(
+            "apply_treatment is not set but %d treatment(s) were returned.",
+            len(treatments),
+        )
+
+    return AdviceResult(
+        full_advice=full_advice,
+        advice_summary=summary,
+        apply_treatment=apply_treatment,
+        treatments=treatments,
+        dosage=dosage,
+        apply_date=apply_date,
+    )
+
+
+def insert_advice(
+    conn: pyodbc.Connection,
+    *,
+    run_date: date,
+    field_id: int,
+    disease_english: str,
+    sent_information: dict[str, Any],
+    result: AdviceResult,
+) -> None:
+    """
+    Store the advisory, replacing any earlier row for the same day/field/disease.
+
+    The table's primary key is (date, field_id, disease), so an explicit
+    delete keeps re-runs of the same day from violating it instead of
+    accumulating duplicates. The
+    delete and insert run inside one transaction: with a single sequential
+    runner, autocommitting each statement separately was harmless, but the
+    batch pipeline writes from several worker threads at once, and a crash
+    between the two statements would otherwise leave that field's advisory
+    silently missing for the day.
+    """
+    treatments = _json_column(result.treatments, "treatments", _TREATMENTS_MAX_CHARS)
+    dosage = _json_column(result.dosage, "dosage", _DOSAGE_MAX_CHARS)
+    sent = json.dumps(sent_information, ensure_ascii=False, default=_json_default)
+
+    with transaction(conn):
+        execute(conn, _DELETE_ADVICE_SQL, (run_date, field_id, disease_english))
+        execute(
+            conn,
+            _INSERT_ADVICE_SQL,
+            (
+                run_date,
+                field_id,
+                disease_english,
+                sent,
+                result.full_advice,
+                result.advice_summary,
+                1 if result.apply_treatment else 0,
+                treatments,
+                dosage,
+                result.apply_date,
+            ),
+        )
+    logger.info(
+        "Stored advice for field %s / %s on %s (apply_treatment=%s).",
+        field_id,
+        disease_english,
+        run_date.isoformat(),
+        int(result.apply_treatment),
+    )
+
+
+def _json_column(values: list[str], column: str, max_chars: int) -> str | None:
+    """Serialise a list column, using NULL for an empty list as the spec requires."""
+    if not values:
+        return None
+    encoded = json.dumps(values, ensure_ascii=False)
+    if len(encoded) > max_chars:
+        raise PipelineError(
+            f"advice.{column} would need {len(encoded)} characters but the column "
+            f"holds {max_chars}: {encoded}"
+        )
+    return encoded
+
+
+def _json_default(obj: Any) -> Any:
+    if isinstance(obj, date):
+        return obj.isoformat()
+    raise TypeError(f"Object of type {type(obj)!r} is not JSON serializable")
+
+
+def _load_json_object(raw: str) -> dict[str, Any]:
+    text = (raw or "").strip()
+    if not text:
+        raise PipelineError("Advice LLM returned an empty response.")
+
+    fenced = re.search(r"```(?:json)?\s*(.*?)\s*```", text, flags=re.DOTALL | re.IGNORECASE)
+    if fenced:
+        text = fenced.group(1).strip()
+
+    try:
+        parsed = json.loads(text)
+    except json.JSONDecodeError:
+        match = re.search(r"\{.*\}", text, flags=re.DOTALL)
+        if match is None:
+            raise PipelineError("Advice LLM response is not valid JSON.") from None
+        try:
+            parsed = json.loads(match.group(0))
+        except json.JSONDecodeError as exc:
+            raise PipelineError("Advice LLM response is not valid JSON.") from exc
+
+    if not isinstance(parsed, dict):
+        raise PipelineError("Advice LLM response is not a JSON object.")
+    return parsed
+
+
+def _require_text(payload: dict[str, Any], key: str) -> str:
+    value = payload.get(key)
+    if not isinstance(value, str) or not value.strip():
+        raise PipelineError(f"Advice LLM response is missing a non-empty '{key}'.")
+    return value.strip()
+
+
+def _string_list(value: Any) -> list[str]:
+    if value is None:
+        return []
+    if isinstance(value, str):
+        stripped = value.strip()
+        return [stripped] if stripped else []
+    if isinstance(value, list):
+        return [str(item).strip() for item in value if str(item).strip()]
+    return [str(value).strip()]
+
+
+def _to_bool(value: Any) -> bool:
+    if isinstance(value, bool):
+        return value
+    if isinstance(value, (int, float)):
+        return bool(value)
+    if isinstance(value, str):
+        return value.strip().lower() in {"1", "true", "yes", "si", "sì"}
+    return False
+
+
+def _to_date(value: Any) -> date | None:
+    if value is None:
+        return None
+    if isinstance(value, date):
+        return value
+    text = str(value).strip()
+    if not text or text.lower() in {"null", "none", ""}:
+        return None
+    for fmt in ("%Y-%m-%d", "%d-%m-%Y", "%d/%m/%Y"):
+        try:
+            return datetime.strptime(text, fmt).date()
+        except ValueError:
+            continue
+    logger.warning("Could not parse apply_date '%s'; storing NULL.", text)
+    return None

pipeline/stages/advice_context.py

@@ -0,0 +1,129 @@
+"""Part_Two, step 2.3: fold products and the previous advice into json_for_advice_generation."""
+
+from __future__ import annotations
+
+import copy
+import logging
+from datetime import date
+from typing import Any, Iterable
+
+import pyodbc
+
+from pipeline.db import fetch_one
+from pipeline.jsonutils import parse_json_list
+from pipeline.stages.product_json import ProductJsonBuilder, normalize_name
+from pipeline.stages.vector import RecommendedProduct
+
+logger = logging.getLogger(__name__)
+
+_PRODUCT_LIST_KEYS = ("allowed_products", "applied_treatment", "suggested_products")
+
+_LAST_ADVICE_SQL = """
+SELECT TOP 1
+    advice_summary,
+    treatments
+FROM advice
+WHERE field_id = ?
+  AND disease = ?
+  AND [date] < ?
+ORDER BY [date] DESC
+"""
+
+
+def load_last_advice(
+    conn: pyodbc.Connection,
+    field_id: int,
+    disease_english: str,
+    as_of: date,
+) -> tuple[str, list[str]]:
+    """Return (advice_summary, treatment names) of the most recent advice before as_of."""
+    row = fetch_one(conn, _LAST_ADVICE_SQL, (field_id, disease_english, as_of))
+    if row is None:
+        logger.info(
+            "No previous advice for field %s / %s before %s.",
+            field_id,
+            disease_english,
+            as_of.isoformat(),
+        )
+        return "", []
+
+    summary = (row.advice_summary or "").strip()
+    treatments = [name.strip() for name in parse_json_list(row.treatments) if name.strip()]
+    logger.info("Loaded previous advice with %d suggested product(s).", len(treatments))
+    return summary, treatments
+
+
+def enrich_advice_json(
+    advice_json: dict[str, Any],
+    *,
+    conn: pyodbc.Connection,
+    builder: ProductJsonBuilder,
+    recommended: Iterable[RecommendedProduct],
+    last_advice_summary: str,
+    last_advice_treatments: Iterable[str],
+) -> None:
+    """
+    Add allowed_products and last_advice, and expand applied_treatment in place.
+
+    A product already described under allowed_products is referenced by name
+    everywhere else; any other product is embedded as a full Product JSON so the
+    LLM always has the label information it needs. `conn` is the calling
+    thread's own SQL connection; `builder` may be shared with other threads
+    handling other fields for the same crop/disease pair (see
+    `ProductJsonBuilder`), so it never stores a connection itself.
+    """
+    recommended = list(recommended)
+
+    advice_json["allowed_products"] = [
+        builder.build(conn, item.product_name) for item in recommended
+    ]
+    vector_names = {normalize_name(item.product_name) for item in recommended}
+
+    applied_names: set[str] = set()
+    for day in advice_json.get("meteorological_data", []):
+        names = day.get("applied_treatment") or []
+        applied_names.update(normalize_name(name) for name in names)
+        day["applied_treatment"] = [
+            name if normalize_name(name) in vector_names else builder.build(conn, name)
+            for name in names
+        ]
+
+    known_names = vector_names | applied_names
+    advice_json["last_advice"] = {
+        "advice_summary": last_advice_summary,
+        "suggested_products": [
+            name if normalize_name(name) in known_names else builder.build(conn, name)
+            for name in last_advice_treatments
+        ],
+    }
+
+
+def _collapse_entry(entry: Any) -> Any:
+    """Reduce a Product JSON object to its product name; leave plain names untouched."""
+    if isinstance(entry, dict) and len(entry) == 1:
+        return next(iter(entry))
+    return entry
+
+
+def collapse_product_json(advice_json: dict[str, Any]) -> dict[str, Any]:
+    """
+    Return a copy where every embedded Product JSON is replaced by its name.
+
+    This is the version stored in advice.sent_information: the same payload the
+    LLM received, minus the product label detail.
+    """
+    collapsed = copy.deepcopy(advice_json)
+    _collapse_in_place(collapsed)
+    return collapsed
+
+
+def _collapse_in_place(node: Any) -> None:
+    if isinstance(node, dict):
+        for key, value in node.items():
+            if key in _PRODUCT_LIST_KEYS and isinstance(value, list):
+                node[key] = [_collapse_entry(item) for item in value]
+            else:
+                _collapse_in_place(value)
+    elif isinstance(node, list):
+        for item in node:
+            _collapse_in_place(item)

pipeline/stages/assemble.py

@@ -0,0 +1,104 @@
+"""Assemble the advice-generation JSON and the query-synthesis JSON."""
+
+from __future__ import annotations
+
+from datetime import date
+from typing import Any
+
+from pipeline.window import build_window, format_date, today_and_future
+
+
+METRIC_KEYS = (
+    "min_air_temperature_c",
+    "max_air_temperature_c",
+    "average_air_temperature_c",
+    "min_relative_humidity_percent",
+    "max_relative_humidity_percent",
+    "average_relative_humidity_percent",
+    "rainfall_mm",
+    "leaf_wetness_minutes",
+    "potential_evapotranspiration_mm",
+)
+
+
+def _day_entry(
+    day: date,
+    as_of: date,
+    weather: dict[date, dict[str, Any]],
+    phenology: dict[date, str | None],
+    forecasts: dict[date, str | None],
+    applied_treatments: dict[date, list[str]],
+) -> dict[str, Any]:
+    metrics = weather.get(day, {})
+    entry: dict[str, Any] = {"date": format_date(day)}
+    for key in METRIC_KEYS:
+        entry[key] = metrics.get(key)
+    entry["phenology_phase"] = phenology.get(day)
+    entry["disease_forecast"] = forecasts.get(day)
+    # Treatments can only have been applied on or before as_of; future days
+    # have no applied treatment, so the field is omitted for them.
+    if day <= as_of:
+        entry["applied_treatment"] = list(applied_treatments.get(day, []))
+    return entry
+
+
+def assemble_json_for_advice_generation(
+    as_of: date,
+    weather: dict[date, dict[str, Any]],
+    phenology: dict[date, str | None],
+    forecasts: dict[date, str | None],
+    applied_treatments: dict[date, list[str]] | None = None,
+) -> dict[str, Any]:
+    """Build meteorological_data (as_of-5..as_of) and weather_forecasts (as_of+1..as_of+5)."""
+    window = build_window(as_of)
+    applied = applied_treatments or {}
+    meteorological = [
+        _day_entry(day, as_of, weather, phenology, forecasts, applied)
+        for day in window
+        if day <= as_of
+    ]
+    forecasts_section = [
+        _day_entry(day, as_of, weather, phenology, forecasts, applied)
+        for day in window
+        if day > as_of
+    ]
+    return {
+        "meteorological_data": meteorological,
+        "weather_forecasts": forecasts_section,
+    }
+
+
+def build_json_for_query_synthesis(
+    as_of: date,
+    crop_english: str,
+    disease_english: str,
+    weather: dict[date, dict[str, Any]],
+    phenology: dict[date, str | None],
+    first_future_disease_date: date | None,
+) -> dict[str, Any]:
+    """
+    Build the JSON sent to the LLM for query synthesis.
+
+    If a disease is predicted in upcoming days: include as_of .. first future disease date.
+    Otherwise: include as_of .. as_of+5.
+    """
+    if first_future_disease_date is not None:
+        end = first_future_disease_date
+    else:
+        end = as_of.fromordinal(as_of.toordinal() + 5)
+
+    days = [d for d in today_and_future(as_of) if d <= end]
+    weather_forecasts: list[dict[str, Any]] = []
+    for day in days:
+        metrics = weather.get(day, {})
+        entry: dict[str, Any] = {"date": format_date(day)}
+        for key in METRIC_KEYS:
+            entry[key] = metrics.get(key)
+        weather_forecasts.append(entry)
+
+    return {
+        "crop": crop_english,
+        "disease": disease_english,
+        "phenology_phase": phenology.get(as_of),
+        "weather_forecasts": weather_forecasts,
+    }

pipeline/stages/disease.py

@@ -0,0 +1,172 @@
+"""Step 2: disease forecast model lookup and per-day FASE resolution."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import date
+
+import pyodbc
+
+from pipeline.db import fetch_all
+from pipeline.errors import NoModelConfiguredError
+from pipeline.window import build_window
+
+
+@dataclass(frozen=True)
+class DiseaseForecast:
+    anmod_id: int
+    disease_raw: str
+    disease_english: str
+    station: int | None
+    forecasts_by_day: dict[date, str | None]
+    first_future_disease_date: date | None
+    has_risk: bool
+
+
+_MODEL_SQL = """
+SELECT anmod_id, anmod_model, anmod_enabled
+FROM AI_agrosupport_agro_models
+WHERE anmod_cmplay = ?
+"""
+
+_TMP2_SQL = """
+SELECT
+    CONVERT(date, model_date) AS model_day,
+    model_description,
+    model_station
+FROM AI_agrosupport_agro_model_tmp2
+WHERE model_model = ?
+  AND CONVERT(date, model_date) >= ?
+  AND CONVERT(date, model_date) <= ?
+"""
+
+
+def _resolve_day_fase(descriptions: set[str]) -> str | None:
+    """FASE5 beats FASE2; anything else is ignored."""
+    upper = {d.upper() for d in descriptions if d}
+    if "FASE5" in upper:
+        return "FASE5"
+    if "FASE2" in upper:
+        return "FASE2"
+    return None
+
+
+def find_anmod_id(conn: pyodbc.Connection, field_id: int, disease_name: str) -> int:
+    """
+    Look up the anmod_id for a field's disease model by name.
+
+    Used by the single-field `one` CLI path, where `anmod_id` is not known
+    ahead of time. The batch worklist resolves it directly with one join per
+    crop-disease pair instead (see `pipeline.worklist.build_jobs`), so it
+    never calls this per field.
+    """
+    models = fetch_all(conn, _MODEL_SQL, (field_id,))
+    target = disease_name.strip().upper()
+    match = next(
+        (row for row in models if (row.anmod_model or "").strip().upper() == target),
+        None,
+    )
+    if match is None:
+        raise NoModelConfiguredError(
+            f"No disease forecast model is configured for {disease_name}."
+        )
+    return int(match.anmod_id)
+
+
+def resolve_disease_forecast(
+    conn: pyodbc.Connection,
+    field_id: int,
+    disease_name: str,
+    disease_english: str,
+    as_of: date,
+    fallback_station: int | None,
+) -> DiseaseForecast:
+    """Find the disease model for the field and build the 11-day forecast list."""
+    anmod_id = find_anmod_id(conn, field_id, disease_name)
+    return resolve_disease_forecast_by_anmod(
+        conn,
+        anmod_id=anmod_id,
+        disease_name=disease_name,
+        disease_english=disease_english,
+        as_of=as_of,
+        fallback_station=fallback_station,
+    )
+
+
+def resolve_disease_forecast_by_anmod(
+    conn: pyodbc.Connection,
+    anmod_id: int,
+    disease_name: str,
+    disease_english: str,
+    as_of: date,
+    fallback_station: int | None,
+) -> DiseaseForecast:
+    """Build the 11-day forecast list for an already-known anmod_id."""
+    window = build_window(as_of)
+    start, end = window[0], window[-1]
+    rows = fetch_all(conn, _TMP2_SQL, (anmod_id, start, end))
+
+    by_day: dict[date, set[str]] = {d: set() for d in window}
+    stations: list[int] = []
+    for row in rows:
+        day = row.model_day
+        if hasattr(day, "date"):
+            day = day.date()
+        day = date(day.year, day.month, day.day)
+        if day not in by_day:
+            continue
+        if row.model_description:
+            by_day[day].add(str(row.model_description).strip())
+        if row.model_station is not None:
+            stations.append(int(row.model_station))
+
+    forecasts: dict[date, str | None] = {
+        day: _resolve_day_fase(descs) for day, descs in by_day.items()
+    }
+
+    first_future: date | None = None
+    for day in sorted(d for d in forecasts if d > as_of):
+        if forecasts[day] is not None:
+            first_future = day
+            break
+
+    station = stations[0] if stations else fallback_station
+    has_risk = any(v is not None for v in forecasts.values())
+
+    return DiseaseForecast(
+        anmod_id=anmod_id,
+        disease_raw=disease_name,
+        disease_english=disease_english,
+        station=station,
+        forecasts_by_day=forecasts,
+        first_future_disease_date=first_future,
+        has_risk=has_risk,
+    )
+
+
+def nearest_fase(
+    forecasts_by_day: dict[date, str | None],
+    as_of: date,
+) -> str | None:
+    """
+    Find the FASE closest to as_of, preferring today/future over past.
+
+    Used by the product prefilter (Part_One, step 3).
+    """
+    if forecasts_by_day.get(as_of) is not None:
+        return forecasts_by_day[as_of]
+
+    future = sorted(
+        (d for d, v in forecasts_by_day.items() if d > as_of and v is not None),
+    )
+    if future:
+        return forecasts_by_day[future[0]]
+
+    past = sorted(
+        (d for d, v in forecasts_by_day.items() if d < as_of and v is not None),
+        reverse=True,
+    )
+    if past:
+        return forecasts_by_day[past[0]]
+
+    return None

pipeline/stages/field.py

@@ -0,0 +1,68 @@
+"""Step 1: resolve cultivated crop and organic/non-organic flag."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import pyodbc
+
+from pipeline.db import fetch_one
+from pipeline.errors import PipelineError
+from pipeline.vocab import Vocabulary
+
+
+@dataclass(frozen=True)
+class FieldInfo:
+    field_id: int
+    crop_italian: str
+    crop_english: str
+    organic: bool
+    cmplay_station: int | None
+    cmplay_colture: int
+
+
+_LAYER_SQL = """
+SELECT cmplay_id, cmplay_colture, cmplay_imp, cmplay_station
+FROM AI_agrosupport_cmp_layers
+WHERE cmplay_id = ?
+"""
+
+_COLTURE_SQL = """
+SELECT colture_id, colture_name
+FROM AI_agrosupport_an_colture
+WHERE colture_id = ?
+"""
+
+
+def resolve_field(
+    conn: pyodbc.Connection,
+    field_id: int,
+    crop_vocab: Vocabulary,
+) -> FieldInfo:
+    """Look up the crop cultivated on the field and whether it is organic."""
+    layer = fetch_one(conn, _LAYER_SQL, (field_id,))
+    if layer is None:
+        raise PipelineError(f"Field ID {field_id} not found in AI_agrosupport_cmp_layers.")
+
+    colture_id = layer.cmplay_colture
+    if colture_id is None:
+        raise PipelineError(f"Field ID {field_id} has no cmplay_colture value.")
+
+    colture = fetch_one(conn, _COLTURE_SQL, (colture_id,))
+    if colture is None:
+        raise PipelineError(
+            f"colture_id {colture_id} for field {field_id} not found in AI_agrosupport_an_colture."
+        )
+
+    italian = (colture.colture_name or "").strip()
+    english = crop_vocab.to_english(italian, kind="crop")
+    organic = layer.cmplay_imp == 3
+
+    return FieldInfo(
+        field_id=field_id,
+        crop_italian=italian,
+        crop_english=english,
+        organic=organic,
+        cmplay_station=int(layer.cmplay_station) if layer.cmplay_station is not None else None,
+        cmplay_colture=int(colture_id),
+    )

pipeline/stages/llm.py

@@ -0,0 +1,210 @@
+"""LLM provider abstraction and query-list parsing."""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+from typing import Any, Callable
+
+from pipeline.config import LlmSettings, Settings
+from pipeline.errors import PipelineError
+
+logger = logging.getLogger(__name__)
+
+
+def parse_queries(raw: str) -> list[str]:
+    """
+    Parse LLM output into a list of one or two query strings.
+
+    Accepts:
+      1. A raw JSON array of strings
+      2. Markdown-fenced JSON
+      3. QUERY 1: ... / QUERY 2: ... prose
+      4. A single plain-text string as a last resort
+    """
+    text = (raw or "").strip()
+    if not text:
+        raise PipelineError("LLM returned an empty response.")
+
+    # Strip markdown fences if present.
+    fenced = re.search(r"```(?:json)?\s*(.*?)\s*```", text, flags=re.DOTALL | re.IGNORECASE)
+    if fenced:
+        text = fenced.group(1).strip()
+
+    # Try JSON array first.
+    try:
+        parsed = json.loads(text)
+        if isinstance(parsed, list) and all(isinstance(item, str) for item in parsed):
+            queries = [item.strip() for item in parsed if item.strip()]
+            if queries:
+                return queries[:2]
+    except json.JSONDecodeError:
+        pass
+
+    # Try to locate a JSON array substring.
+    match = re.search(r"\[.*\]", text, flags=re.DOTALL)
+    if match:
+        try:
+            parsed = json.loads(match.group(0))
+            if isinstance(parsed, list) and all(isinstance(item, str) for item in parsed):
+                queries = [item.strip() for item in parsed if item.strip()]
+                if queries:
+                    return queries[:2]
+        except json.JSONDecodeError:
+            pass
+
+    # QUERY n: prose split.
+    parts = re.split(r"(?i)\bQUERY\s*\d+\s*:\s*", text)
+    queries = [part.strip() for part in parts if part.strip()]
+    if len(queries) >= 2:
+        return queries[:2]
+    if len(queries) == 1 and re.search(r"(?i)\bQUERY\s*\d+\s*:", raw or ""):
+        return queries[:1]
+
+    # Single-string fallback.
+    return [text]
+
+
+def synthesize_queries(
+    settings: Settings,
+    system_prompt: str,
+    subset: dict[str, Any],
+    llm_call: Callable[..., str] = None,
+) -> list[str]:
+    """
+    Call the configured LLM provider and return one or two search queries.
+
+    `llm_call` defaults to the plain, unguarded `call_llm` (fine for the
+    single-field `one` CLI path). The batch orchestrator passes
+    `resources.call_llm` instead, which adds the shared Gemini semaphore,
+    rate limiting, and retry policy around the same call signature.
+    """
+    llm_call = llm_call or call_llm
+    llm_cfg = settings.llm_query_synthesis
+    user_content = json.dumps(subset, ensure_ascii=False, indent=2)
+
+    raw = llm_call(settings, llm_cfg, system_prompt, user_content)
+
+    queries = parse_queries(raw)
+    logger.info("LLM produced %d query/queries", len(queries))
+    return queries
+
+
+def call_llm(
+    settings: Settings,
+    llm_cfg: LlmSettings,
+    system_prompt: str,
+    user_content: str,
+    json_output: bool = False,
+) -> str:
+    """Dispatch a system/user prompt pair to the provider configured in llm_cfg."""
+    provider = llm_cfg.provider
+
+    logger.info("Calling LLM provider=%s model=%s", provider, _model_name(llm_cfg))
+    logger.debug(
+        "LLM REQUEST (system_prompt + JSON user content):\n--- SYSTEM PROMPT ---\n%s\n--- USER JSON ---\n%s",
+        system_prompt,
+        user_content,
+    )
+    if provider == "gemini":
+        raw = _call_gemini(settings, llm_cfg, system_prompt, user_content, json_output)
+    elif provider == "openai":
+        raw = _call_openai(settings, llm_cfg, system_prompt, user_content, json_output)
+    elif provider == "anthropic":
+        raw = _call_anthropic(settings, llm_cfg, system_prompt, user_content)
+    else:
+        raise PipelineError(f"Unsupported LLM provider: {provider}")
+
+    logger.debug("LLM RESPONSE (raw text):\n%s", raw)
+    return raw
+
+
+def _model_name(llm_cfg: LlmSettings) -> str:
+    return {
+        "gemini": llm_cfg.gemini_model,
+        "openai": llm_cfg.openai_model,
+        "anthropic": llm_cfg.anthropic_model,
+    }.get(llm_cfg.provider, "?")
+
+
+def _call_gemini(
+    settings: Settings,
+    llm_cfg: LlmSettings,
+    system_prompt: str,
+    user_content: str,
+    json_output: bool = False,
+) -> str:
+    if not settings.gemini_api_key:
+        raise PipelineError("GEMINI_API_KEY is required for the gemini LLM provider.")
+    from google import genai
+    from google.genai import types
+
+    client = genai.Client(api_key=settings.gemini_api_key)
+    response = client.models.generate_content(
+        model=llm_cfg.gemini_model,
+        contents=user_content,
+        config=types.GenerateContentConfig(
+            system_instruction=system_prompt,
+            temperature=llm_cfg.temperature,
+            max_output_tokens=llm_cfg.max_tokens,
+            response_mime_type="application/json" if json_output else None,
+        ),
+    )
+    text = getattr(response, "text", None)
+    if not text:
+        raise PipelineError("Gemini returned no text content.")
+    return text
+
+
+def _call_openai(
+    settings: Settings,
+    llm_cfg: LlmSettings,
+    system_prompt: str,
+    user_content: str,
+    json_output: bool = False,
+) -> str:
+    if not settings.openai_api_key:
+        raise PipelineError("OPENAI_API_KEY is required for the openai LLM provider.")
+    from openai import OpenAI
+
+    client = OpenAI(api_key=settings.openai_api_key)
+    extra: dict[str, Any] = {}
+    if json_output:
+        extra["response_format"] = {"type": "json_object"}
+    response = client.chat.completions.create(
+        model=llm_cfg.openai_model,
+        temperature=llm_cfg.temperature,
+        max_tokens=llm_cfg.max_tokens,
+        messages=[
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_content},
+        ],
+        **extra,
+    )
+    text = response.choices[0].message.content
+    if not text:
+        raise PipelineError("OpenAI returned no text content.")
+    return text
+
+
+def _call_anthropic(
+    settings: Settings, llm_cfg: LlmSettings, system_prompt: str, user_content: str
+) -> str:
+    if not settings.anthropic_api_key:
+        raise PipelineError("ANTHROPIC_API_KEY is required for the anthropic LLM provider.")
+    from anthropic import Anthropic
+
+    client = Anthropic(api_key=settings.anthropic_api_key)
+    response = client.messages.create(
+        model=llm_cfg.anthropic_model,
+        max_tokens=llm_cfg.max_tokens,
+        temperature=llm_cfg.temperature,
+        system=system_prompt,
+        messages=[{"role": "user", "content": user_content}],
+    )
+    parts = [block.text for block in response.content if getattr(block, "type", None) == "text"]
+    text = "\n".join(parts).strip()
+    if not text:
+        raise PipelineError("Anthropic returned no text content.")
+    return text

pipeline/stages/phenology.py

@@ -0,0 +1,100 @@
+"""Step 3: phenology phase resolution with carry-forward."""
+
+from __future__ import annotations
+
+from datetime import date
+from typing import Any
+
+import pyodbc
+
+from pipeline.db import fetch_all
+from pipeline.window import build_window
+
+
+_FENO_SQL = """
+SELECT
+    rilfeno_date,
+    rilfeno_class1,
+    rilfeno_class2,
+    rilfeno_class3
+FROM AI_agrosupport_agro_ril_feno
+WHERE rilfeno_layer = ?
+  AND rilfeno_date <= ?
+ORDER BY rilfeno_date ASC
+"""
+
+_PHASE_SQL = """
+SELECT ff_id, ff_name
+FROM AI_agrosupport_an_pheno_phase
+WHERE ff_id IN ({placeholders})
+"""
+
+
+def _to_date(value: Any) -> date:
+    if hasattr(value, "date") and callable(value.date):
+        value = value.date()
+    return date(value.year, value.month, value.day)
+
+
+def _pick_class(row: pyodbc.Row) -> int | None:
+    if row.rilfeno_class3 is not None:
+        return int(row.rilfeno_class3)
+    if row.rilfeno_class2 is not None:
+        return int(row.rilfeno_class2)
+    if row.rilfeno_class1 is not None:
+        return int(row.rilfeno_class1)
+    return None
+
+
+def load_phenology(
+    conn: pyodbc.Connection,
+    field_id: int,
+    as_of: date,
+) -> dict[date, str | None]:
+    """
+    Build phenology_phase per day in the 11-day window.
+
+    Observations with rilfeno_date <= as_of are carried forward to each day
+    (latest observation at or before that day). Future days are always null.
+    Today's phase equals yesterday's (which, under carry-forward, is the same
+    observation as of as_of-1, or the latest available before as_of).
+    """
+    window = build_window(as_of)
+    rows = fetch_all(conn, _FENO_SQL, (field_id, as_of))
+
+    observations: list[tuple[date, int]] = []
+    ff_ids: set[int] = set()
+    for row in rows:
+        day = _to_date(row.rilfeno_date)
+        ff_id = _pick_class(row)
+        if ff_id is None:
+            continue
+        observations.append((day, ff_id))
+        ff_ids.add(ff_id)
+
+    name_by_id: dict[int, str] = {}
+    if ff_ids:
+        placeholders = ",".join("?" for _ in ff_ids)
+        sql = _PHASE_SQL.format(placeholders=placeholders)
+        for phase in fetch_all(conn, sql, tuple(ff_ids)):
+            name_by_id[int(phase.ff_id)] = str(phase.ff_name).strip()
+
+    result: dict[date, str | None] = {d: None for d in window}
+    for day in window:
+        if day > as_of:
+            result[day] = None
+            continue
+        # Carry forward the latest observation at or before this day.
+        applicable = [obs for obs in observations if obs[0] <= day]
+        if not applicable:
+            result[day] = None
+            continue
+        _, ff_id = applicable[-1]
+        result[day] = name_by_id.get(ff_id)
+
+    # Explicit rule: today's phase equals yesterday's.
+    yesterday = as_of.fromordinal(as_of.toordinal() - 1)
+    if yesterday in result:
+        result[as_of] = result[yesterday]
+
+    return result

pipeline/stages/product_json.py

@@ -0,0 +1,162 @@
+"""Part_Two, step 2: build the full Product JSON for a product name."""
+
+from __future__ import annotations
+
+import logging
+import threading
+from decimal import Decimal
+from typing import Any
+
+import pyodbc
+
+from pipeline.db import fetch_all
+from pipeline.jsonutils import normalize_name, parse_json_list
+from pipeline.stages.products import ProductIndex
+
+logger = logging.getLogger(__name__)
+
+
+_USES_SQL = """
+SELECT
+    dose_min,
+    dose_max,
+    dose_unit,
+    concentration_min,
+    concentration_max,
+    concentration_unit,
+    treatment_interval_min_days,
+    treatment_interval_max_days,
+    max_treatments_per_season,
+    pre_harvest_interval_days,
+    growth_stage_start,
+    growth_stage_end
+FROM product_uses
+WHERE product_id = ?
+  AND crop = ?
+  AND disease = ?
+ORDER BY id ASC
+"""
+
+_CHUNKS_SQL = """
+SELECT
+    chunk_type,
+    chunk_text,
+    target_crops,
+    target_diseases
+FROM label_chunks
+WHERE product_id = ?
+ORDER BY chunk_id ASC
+"""
+
+_USE_KEYS = (
+    "dose_min",
+    "dose_max",
+    "dose_unit",
+    "concentration_min",
+    "concentration_max",
+    "concentration_unit",
+    "treatment_interval_min_days",
+    "treatment_interval_max_days",
+    "max_treatments_per_season",
+    "pre_harvest_interval_days",
+    "growth_stage_start",
+    "growth_stage_end",
+)
+
+
+def _scalar(value: Any) -> Any:
+    if isinstance(value, Decimal):
+        return float(value)
+    if isinstance(value, str):
+        return value.strip()
+    return value
+
+
+class ProductJsonBuilder:
+    """
+    Build Product JSON objects for a fixed crop/disease context.
+
+    Results are cached per normalized product name, since the same product can
+    appear in the vector search results, the applied treatments, and the
+    previous advice's suggested products. In the batch pipeline one builder is
+    shared across every field growing the same crop with the same disease, so
+    it holds no SQL connection of its own — pyodbc connections are not safe to
+    use from multiple threads at once, and a shared builder can be called from
+    whichever worker thread is currently handling a field for this pair. Each
+    call supplies that thread's own connection instead.
+    """
+
+    def __init__(
+        self,
+        product_index: ProductIndex,
+        crop_english: str,
+        disease_english: str,
+    ) -> None:
+        self._products = product_index
+        self._crop = crop_english
+        self._disease = disease_english
+        self._cache: dict[str, dict[str, Any] | str] = {}
+        self._cache_lock = threading.Lock()
+
+    def build(self, conn: pyodbc.Connection, product_name: str) -> dict[str, Any] | str:
+        """
+        Return {product_name: {...}} for a known product.
+
+        Products absent from the curated `products` table (for instance names
+        coming from the operations catalog) cannot be described, so the plain
+        name is returned instead.
+        """
+        name = product_name.strip()
+        key = normalize_name(name)
+        with self._cache_lock:
+            cached = self._cache.get(key)
+        if cached is not None:
+            return cached
+
+        result = self._build_uncached(conn, name)
+        with self._cache_lock:
+            self._cache[key] = result
+        return result
+
+    def _build_uncached(self, conn: pyodbc.Connection, name: str) -> dict[str, Any] | str:
+        row = self._products.by_name(name)
+        if row is None:
+            logger.warning(
+                "Product '%s' has no row in products; keeping it as a plain name.",
+                name,
+            )
+            return name
+
+        return {
+            row.product_name: {
+                "active_ingredients": row.active_ingredients,
+                "frac_groups": row.frac_groups,
+                "systemicity": row.systemicity,
+                "uses": self._load_uses(conn, row.product_id),
+                "information_chunks": self._load_chunks(conn, row.product_id),
+            }
+        }
+
+    def _load_uses(self, conn: pyodbc.Connection, product_id: str) -> list[dict[str, Any]]:
+        rows = fetch_all(conn, _USES_SQL, (product_id, self._crop, self._disease))
+        return [
+            {key: _scalar(getattr(row, key)) for key in _USE_KEYS}
+            for row in rows
+        ]
+
+    def _load_chunks(self, conn: pyodbc.Connection, product_id: str) -> list[dict[str, Any]]:
+        rows = fetch_all(conn, _CHUNKS_SQL, (product_id,))
+        chunks: list[dict[str, Any]] = []
+        for row in rows:
+            if self._crop not in parse_json_list(row.target_crops):
+                continue
+            diseases = parse_json_list(row.target_diseases)
+            if diseases and self._disease not in diseases:
+                continue
+            chunks.append(
+                {
+                    "chunk_type": str(row.chunk_type).strip(),
+                    "chunk_text": str(row.chunk_text),
+                }
+            )
+        return chunks

pipeline/stages/products.py

@@ -0,0 +1,164 @@
+"""SQL product catalog: a load-once, read-only index shared by every job in a batch."""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+
+import pyodbc
+
+from pipeline.db import fetch_all
+from pipeline.jsonutils import normalize_name, parse_json_list
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class ProductCandidate:
+    product_id: str
+    product_name: str
+    registration_number: str | None
+
+
+@dataclass(frozen=True)
+class ProductRow:
+    """One `products` row with its JSON columns already parsed and normalised."""
+
+    product_id: str
+    product_name: str
+    registration_number: str | None
+    organic_certified: bool
+    systemicity: str  # stripped, original casing (used verbatim in the Product JSON payload)
+    preventive_action: bool
+    eradicant_action: bool
+    target_crops: list[str]
+    target_diseases: list[str]
+    active_ingredients: list[str]
+    frac_groups: list[str]
+
+    def as_candidate(self) -> ProductCandidate:
+        return ProductCandidate(
+            product_id=self.product_id,
+            product_name=self.product_name,
+            registration_number=self.registration_number,
+        )
+
+
+_PRODUCTS_SQL = """
+SELECT
+    product_id,
+    product_name,
+    registration_number,
+    organic_certified,
+    systemicity,
+    preventive_action,
+    eradicant_action,
+    target_crops,
+    target_diseases,
+    active_ingredients,
+    frac_groups
+FROM products
+"""
+
+
+class ProductIndex:
+    """
+    Read-only, in-memory view of the `products` table, built once per batch.
+
+    The previous per-field implementation ran `SELECT * FROM products` (no
+    WHERE clause) and re-parsed every row's JSON columns on every field. At a
+    few hundred fields and a catalog heading past 600 rows that is hundreds of
+    thousands of redundant `json.loads` calls per morning. This index scans
+    the table once and precomputes two lookups instead:
+
+    - `candidates_for(crop, disease)`: the small bucket of products whose
+      `target_crops` / `target_diseases` include that pair, for the prefilter.
+    - `by_name(product_name)`: the catalog row for a product name, so
+      `ProductJsonBuilder` no longer needs its own
+      `UPPER(LTRIM(RTRIM(product_name))) = UPPER(?)` query (a predicate no
+      index can serve).
+    """
+
+    def __init__(self, rows: list[ProductRow]) -> None:
+        self._by_pair: dict[tuple[str, str], list[ProductRow]] = {}
+        self._by_name: dict[str, ProductRow] = {}
+        for row in rows:
+            self._by_name[normalize_name(row.product_name)] = row
+            for crop in row.target_crops:
+                for disease in row.target_diseases:
+                    self._by_pair.setdefault((crop, disease), []).append(row)
+        logger.info(
+            "Loaded product index: %d product(s), %d crop/disease bucket(s)",
+            len(rows),
+            len(self._by_pair),
+        )
+
+    @classmethod
+    def load(cls, conn: pyodbc.Connection) -> "ProductIndex":
+        rows = [
+            ProductRow(
+                product_id=str(r.product_id),
+                product_name=str(r.product_name).strip(),
+                registration_number=(
+                    str(r.registration_number) if r.registration_number is not None else None
+                ),
+                organic_certified=bool(r.organic_certified),
+                systemicity=(r.systemicity or "").strip(),
+                preventive_action=bool(r.preventive_action),
+                eradicant_action=bool(r.eradicant_action),
+                target_crops=parse_json_list(r.target_crops),
+                target_diseases=parse_json_list(r.target_diseases),
+                active_ingredients=parse_json_list(r.active_ingredients),
+                frac_groups=parse_json_list(r.frac_groups),
+            )
+            for r in fetch_all(conn, _PRODUCTS_SQL)
+        ]
+        return cls(rows)
+
+    def candidates_for(self, crop_english: str, disease_english: str) -> list[ProductRow]:
+        return self._by_pair.get((crop_english, disease_english), [])
+
+    def by_name(self, product_name: str) -> ProductRow | None:
+        return self._by_name.get(normalize_name(product_name))
+
+
+def prefilter_products(
+    index: ProductIndex,
+    *,
+    crop_english: str,
+    disease_english: str,
+    organic: bool,
+    fase: str | None,
+) -> list[ProductCandidate]:
+    """
+    Filter the (crop, disease) bucket by organic flag and FASE rules.
+
+    FASE2 → systemicity in (systemic, mixed) AND preventive_action = 1
+    FASE5 → eradicant_action = 0
+    """
+    candidates: list[ProductCandidate] = []
+
+    for row in index.candidates_for(crop_english, disease_english):
+        if organic and not row.organic_certified:
+            continue
+
+        if fase == "FASE2":
+            if row.systemicity.lower() not in {"systemic", "mixed"}:
+                continue
+            if not row.preventive_action:
+                continue
+        elif fase == "FASE5":
+            if row.eradicant_action:
+                continue
+
+        candidates.append(row.as_candidate())
+
+    logger.info(
+        "Product prefilter: crop=%s disease=%s organic=%s fase=%s -> %d candidates",
+        crop_english,
+        disease_english,
+        organic,
+        fase,
+        len(candidates),
+    )
+    return candidates

pipeline/stages/treatments.py

@@ -0,0 +1,90 @@
+"""Part_Two, step 1: crop protection products applied on the field in the recent past."""
+
+from __future__ import annotations
+
+import logging
+from datetime import date, timedelta
+from typing import Any
+
+import pyodbc
+
+from pipeline.db import fetch_all
+
+logger = logging.getLogger(__name__)
+
+# rilop_operation == 10 identifies a crop protection treatment.
+_TREATMENT_OPERATION = 10
+
+_APPLIED_TREATMENTS_SQL = """
+SELECT DISTINCT
+    o.rilop_date,
+    c.prodgov_name
+FROM AI_agrosupport_agro_ril_operations_points p
+JOIN AI_agrosupport_agro_ril_operations o
+    ON o.rilop_id = p.riloppoint_rilievo
+JOIN AI_agrosupport_agro_ril_operations_products tp
+    ON tp.trprod_rilievo = o.rilop_id
+JOIN AI_agrosupport_products_catalog c
+    ON c.prodgov_id = tp.trprod_product
+WHERE p.riloppoint_point = ?
+  AND o.rilop_date BETWEEN ? AND ?
+  AND o.rilop_operation = ?
+  AND c.prodgov_name IS NOT NULL
+ORDER BY o.rilop_date ASC, c.prodgov_name ASC
+"""
+
+
+def _to_date(value: Any) -> date:
+    if hasattr(value, "date") and callable(value.date):
+        value = value.date()
+    return date(value.year, value.month, value.day)
+
+
+def load_applied_treatments(
+    conn: pyodbc.Connection,
+    field_id: int,
+    as_of: date,
+) -> dict[date, list[str]]:
+    """
+    Return the product names applied on the field per day over as_of-5 .. as_of.
+
+    Days without a treatment are absent from the mapping. An empty result means
+    no operation matched the field, the date window, or the treatment operation
+    type, in which case every applied_treatment stays empty downstream.
+    """
+    start = as_of - timedelta(days=5)
+    rows = fetch_all(
+        conn,
+        _APPLIED_TREATMENTS_SQL,
+        (field_id, start, as_of, _TREATMENT_OPERATION),
+    )
+
+    if not rows:
+        logger.warning(
+            "No crop protection operations for field %s between %s and %s; "
+            "applied_treatment stays empty.",
+            field_id,
+            start.isoformat(),
+            as_of.isoformat(),
+        )
+        return {}
+
+    by_day: dict[date, list[str]] = {}
+    for row in rows:
+        if row.rilop_date is None:
+            continue
+        day = _to_date(row.rilop_date)
+        name = str(row.prodgov_name).strip()
+        if not name:
+            continue
+        names = by_day.setdefault(day, [])
+        if name not in names:
+            names.append(name)
+
+    logger.info(
+        "Applied treatments for field %s: %d day(s), %d product mention(s)",
+        field_id,
+        len(by_day),
+        sum(len(v) for v in by_day.values()),
+    )
+    return by_day

pipeline/stages/vector.py

@@ -0,0 +1,175 @@
+"""Weaviate near_text search constrained to an allowlisted product set."""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from typing import Any
+
+from pipeline.config import Settings
+from pipeline.errors import PipelineError
+from pipeline.stages.products import ProductCandidate
+
+logger = logging.getLogger(__name__)
+
+COLLECTION = "ProductProfile"
+
+# Above this many allowlisted product_ids, encoding every one of them as an
+# OR filter term gets expensive to build and transmit. Past that size the
+# allowlist is a large fraction of the whole catalog anyway, so falling back
+# to a global over-fetch + Python filter (the original approach) is cheaper
+# and just as correct.
+_MAX_FILTERED_IDS = 200
+
+
+@dataclass(frozen=True)
+class RecommendedProduct:
+    product_name: str
+    registration_number: str | None
+    distance: float | None
+    query_index: int
+    product_id: str
+
+
+def connect_client(settings: Settings) -> Any:
+    """
+    Open a Weaviate client for the batch to hold open and reuse across jobs.
+
+    Replaces the old per-call `weaviate.connect_to_local(...)` /
+    `client.close()` pair in `search_products`: opening a fresh gRPC
+    connection for every field does not scale once a run covers hundreds of
+    fields.
+    """
+    try:
+        import weaviate
+    except ImportError as exc:
+        raise PipelineError("weaviate-client is not installed.") from exc
+
+    headers: dict[str, str] = {}
+    if settings.gemini_api_key:
+        # ProductProfile is vectorised with text2vec-palm (gemini-embedding-001),
+        # so near_text calls draw on the same Gemini quota as the LLM calls.
+        headers["X-Goog-Studio-Api-Key"] = settings.gemini_api_key
+        headers["X-Palm-Api-Key"] = settings.gemini_api_key
+
+    return weaviate.connect_to_local(
+        host=settings.weaviate.host,
+        port=settings.weaviate.http_port,
+        grpc_port=settings.weaviate.grpc_port,
+        headers=headers or None,
+    )
+
+
+def search_products(
+    client: Any,
+    queries: list[str],
+    candidates: list[ProductCandidate],
+) -> list[RecommendedProduct]:
+    """
+    Embed each query and retrieve the nearest ProductProfile objects, scoped
+    server-side to the SQL-prefiltered allowlist.
+
+    One query  → top 6 overall.
+    Two queries → top 3 per query (deduplicated preferring first occurrence).
+
+    `client` is a shared, already-connected Weaviate client (see
+    `pipeline.resources.Resources`); this function does not open or close a
+    connection itself, so it is safe to call from many jobs in a row.
+
+    The allowlist is applied as a `product_id` OR filter before ranking,
+    rather than over-fetching globally and intersecting in Python afterwards.
+    The original code fetched a flat `limit=30` and only worked because the
+    whole `ProductProfile` collection had ~30 objects, so the top 30 by raw
+    similarity were, by construction, the entire catalog. Once the catalog
+    grows past that, a selective allowlist (organic + crop + disease + FASE
+    can easily narrow it to a handful of products) would often be missing
+    entirely from an unfiltered top-30, silently returning zero or too few
+    recommendations instead of raising an error.
+    """
+    if not candidates:
+        logger.warning("No product candidates after prefilter; skipping vector search.")
+        return []
+    if not queries:
+        raise PipelineError("Cannot run vector search without at least one query.")
+
+    from weaviate.classes.query import Filter, MetadataQuery
+
+    allowlist = {c.product_id: c for c in candidates}
+    per_query_limit = 6 if len(queries) == 1 else 3
+
+    query_filter = None
+    fetch_limit = per_query_limit
+    if len(allowlist) <= _MAX_FILTERED_IDS:
+        query_filter = Filter.any_of(
+            [Filter.by_property("product_id").equal(pid) for pid in allowlist]
+        )
+    else:
+        fetch_limit = min(200, max(per_query_limit * 5, 15))
+        logger.warning(
+            "Allowlist has %d product(s), above the %d-id server-side filter "
+            "threshold; falling back to a global over-fetch + Python filter "
+            "for this query.",
+            len(allowlist),
+            _MAX_FILTERED_IDS,
+        )
+
+    collection = client.collections.get(COLLECTION)
+    results: list[RecommendedProduct] = []
+    seen_ids: set[str] = set()
+
+    for query_index, query in enumerate(queries):
+        response = collection.query.near_text(
+            query=query,
+            limit=fetch_limit,
+            filters=query_filter,
+            return_metadata=MetadataQuery(distance=True),
+            return_properties=["product_id", "product_name"],
+        )
+
+        kept = 0
+        for obj in response.objects:
+            props: dict[str, Any] = obj.properties or {}
+            product_id = str(props.get("product_id") or "")
+            if product_id not in allowlist:
+                continue
+            if product_id in seen_ids:
+                continue
+
+            candidate = allowlist[product_id]
+            distance = None
+            if obj.metadata is not None:
+                distance = obj.metadata.distance
+
+            results.append(
+                RecommendedProduct(
+                    product_name=candidate.product_name,
+                    registration_number=candidate.registration_number,
+                    distance=distance,
+                    query_index=query_index,
+                    product_id=product_id,
+                )
+            )
+            seen_ids.add(product_id)
+            kept += 1
+            if kept >= per_query_limit:
+                break
+
+        if kept < per_query_limit:
+            logger.warning(
+                "Vector search query[%d]: only %d/%d allowlisted product(s) "
+                "returned (allowlist size=%d); the catalog may not have enough "
+                "matches for this crop/disease/organic/FASE combination.",
+                query_index,
+                kept,
+                per_query_limit,
+                len(allowlist),
+            )
+        else:
+            logger.info(
+                "Vector search query[%d]: kept %d/%d allowlisted products",
+                query_index,
+                kept,
+                per_query_limit,
+            )
+
+    return results

pipeline/stages/weather.py

@@ -0,0 +1,116 @@
+"""Step 3: historical + forecast weather extraction."""
+
+from __future__ import annotations
+
+from datetime import date
+from typing import Any
+
+import pyodbc
+
+from pipeline.db import fetch_all
+from pipeline.window import past_days, today_and_future
+
+
+_WEATHER_COLUMNS = """
+    CONVERT(date, Data) AS weather_day,
+    TemperaturaAriaMin,
+    TemperaturaAriaMax,
+    TemperaturaAriaMedia,
+    UmiditaRelativaMin,
+    UmiditaRelativaMax,
+    UmiditaRelativaMedia,
+    BagnaturaFogliare,
+    Pioggia,
+    EvapoPot
+"""
+
+_METEO_D_SQL = f"""
+SELECT {_WEATHER_COLUMNS}
+FROM AI_agrosupport_TDatiMeteo_D
+WHERE CodiceStazione = ?
+  AND CONVERT(date, Data) >= ?
+  AND CONVERT(date, Data) <= ?
+"""
+
+_METEO_FRC_SQL = f"""
+SELECT {_WEATHER_COLUMNS}
+FROM AI_agrosupport_TDatiMeteo_D_FRC
+WHERE CodiceStazione = ?
+  AND CONVERT(date, Data) >= ?
+  AND CONVERT(date, Data) <= ?
+"""
+
+
+def _to_date(value: Any) -> date:
+    if hasattr(value, "date") and callable(value.date):
+        value = value.date()
+    return date(value.year, value.month, value.day)
+
+
+def _row_to_metrics(row: pyodbc.Row | None) -> dict[str, Any]:
+    if row is None:
+        return {
+            "min_air_temperature_c": None,
+            "max_air_temperature_c": None,
+            "average_air_temperature_c": None,
+            "min_relative_humidity_percent": None,
+            "max_relative_humidity_percent": None,
+            "average_relative_humidity_percent": None,
+            "rainfall_mm": None,
+            "leaf_wetness_minutes": None,
+            "potential_evapotranspiration_mm": None,
+        }
+    return {
+        "min_air_temperature_c": row.TemperaturaAriaMin,
+        "max_air_temperature_c": row.TemperaturaAriaMax,
+        "average_air_temperature_c": row.TemperaturaAriaMedia,
+        "min_relative_humidity_percent": row.UmiditaRelativaMin,
+        "max_relative_humidity_percent": row.UmiditaRelativaMax,
+        "average_relative_humidity_percent": row.UmiditaRelativaMedia,
+        "rainfall_mm": row.Pioggia,
+        "leaf_wetness_minutes": row.BagnaturaFogliare,
+        "potential_evapotranspiration_mm": row.EvapoPot,
+    }
+
+
+def _index_rows(rows: list[pyodbc.Row]) -> dict[date, pyodbc.Row]:
+    indexed: dict[date, pyodbc.Row] = {}
+    for row in rows:
+        day = _to_date(row.weather_day)
+        # Keep the first row per day if duplicates exist.
+        indexed.setdefault(day, row)
+    return indexed
+
+
+def load_weather(
+    conn: pyodbc.Connection,
+    station: int,
+    as_of: date,
+) -> dict[date, dict[str, Any]]:
+    """
+    Load weather for the 11-day window.
+
+    Past days (as_of-5 .. as_of-1) come from TDatiMeteo_D.
+    Today and future (as_of .. as_of+5) come from TDatiMeteo_D_FRC.
+    """
+    past = past_days(as_of)
+    forward = today_and_future(as_of)
+
+    past_rows: dict[date, pyodbc.Row] = {}
+    if past:
+        past_rows = _index_rows(
+            fetch_all(conn, _METEO_D_SQL, (station, past[0], past[-1]))
+        )
+
+    frc_rows: dict[date, pyodbc.Row] = {}
+    if forward:
+        frc_rows = _index_rows(
+            fetch_all(conn, _METEO_FRC_SQL, (station, forward[0], forward[-1]))
+        )
+
+    result: dict[date, dict[str, Any]] = {}
+    for day in past:
+        result[day] = _row_to_metrics(past_rows.get(day))
+    for day in forward:
+        result[day] = _row_to_metrics(frc_rows.get(day))
+    return result

pipeline/vocab.py

@@ -0,0 +1,49 @@
+"""Italian → English vocabulary normalisation."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import yaml
+
+from pipeline.errors import PipelineError
+
+
+class Vocabulary:
+    """Lookup table loaded from a vocab YAML file."""
+
+    def __init__(self, path: Path) -> None:
+        with path.open(encoding="utf-8") as fh:
+            raw = yaml.safe_load(fh) or {}
+        self.canonical_values: set[str] = set(raw.get("canonical_values", []) or [])
+        normalize = raw.get("normalize", {}) or {}
+        self._normalize: dict[str, str] = {
+            str(key).strip().lower(): str(value).strip() for key, value in normalize.items()
+        }
+
+    def to_english(self, italian_name: str, *, kind: str) -> str:
+        """Map an Italian (or already-English) term to its canonical English form."""
+        key = italian_name.strip().lower()
+        if not key:
+            raise PipelineError(f"Empty {kind} name cannot be normalised.")
+
+        if key in self._normalize:
+            return self._normalize[key]
+
+        # Already a canonical English value (case-insensitive match).
+        for value in self.canonical_values:
+            if value.lower() == key:
+                return value
+
+        raise PipelineError(
+            f"No English mapping found for {kind} '{italian_name}'. "
+            f"Add it to the vocabulary file."
+        )
+
+
+def load_crop_vocab(path: Path) -> Vocabulary:
+    return Vocabulary(path)
+
+
+def load_disease_vocab(path: Path) -> Vocabulary:
+    return Vocabulary(path)

pipeline/window.py

@@ -0,0 +1,25 @@
+"""Date-window helpers for the 11-day pipeline horizon."""
+
+from __future__ import annotations
+
+from datetime import date, timedelta
+
+
+def build_window(as_of: date) -> list[date]:
+    """Return the inclusive date list from as_of-5 through as_of+5."""
+    return [as_of + timedelta(days=offset) for offset in range(-5, 6)]
+
+
+def past_days(as_of: date) -> list[date]:
+    """Days strictly before as_of within the window (as_of-5 .. as_of-1)."""
+    return [as_of + timedelta(days=offset) for offset in range(-5, 0)]
+
+
+def today_and_future(as_of: date) -> list[date]:
+    """as_of through as_of+5 inclusive."""
+    return [as_of + timedelta(days=offset) for offset in range(0, 6)]
+
+
+def format_date(d: date) -> str:
+    """Format a date as DD-MM-YYYY for the JSON payload."""
+    return d.strftime("%d-%m-%Y")

pipeline/worklist.py

@@ -0,0 +1,163 @@
+"""Crop-first job discovery: find every (field, disease) pair to run today.
+
+Unlike the original single-field pipeline, the batch does not start from a
+field ID. It starts from `config.yaml`'s `crops:` list, resolves which fields
+grow each crop, and for each of that crop's configured diseases finds the
+fields that also have a matching disease-forecast model configured
+(`AI_agrosupport_agro_models`). Each match becomes one `Job`.
+"""
+
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+
+import pyodbc
+
+from pipeline.config import CropPlan, WorklistSettings
+from pipeline.db import fetch_all
+from pipeline.errors import PipelineError
+from pipeline.vocab import Vocabulary
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass(frozen=True)
+class Job:
+    """One field/disease pair to run the pipeline for."""
+
+    field_id: int
+    crop_english: str
+    crop_italian: str
+    model_name: str  # e.g. "PERONOSPORA"; matched against anmod_model
+    disease_english: str  # canonical English disease name
+    organic: bool
+    station: int | None
+    anmod_id: int
+
+
+_COLTURE_SQL = "SELECT colture_id, colture_name FROM AI_agrosupport_an_colture"
+
+_JOBS_SQL_TEMPLATE = """
+SELECT
+    l.cmplay_id,
+    l.cmplay_colture,
+    l.cmplay_imp,
+    l.cmplay_station,
+    m.anmod_id,
+    m.anmod_enabled
+FROM AI_agrosupport_cmp_layers l
+JOIN AI_agrosupport_agro_models m ON m.anmod_cmplay = l.cmplay_id
+WHERE l.cmplay_colture IN ({placeholders})
+  AND UPPER(LTRIM(RTRIM(m.anmod_model))) = ?
+"""
+
+
+def _load_crop_to_colture_ids(
+    conn: pyodbc.Connection, crop_vocab: Vocabulary
+) -> dict[str, list[int]]:
+    """
+    Map every canonical English crop name to the colture_ids that resolve to it.
+
+    `Vocabulary.to_english` already strips whitespace and lower-cases before
+    matching, so `AI_agrosupport_an_colture` rows like `"Vite "` (trailing
+    space) resolve correctly without extra cleanup here. Colture names with no
+    vocabulary entry are logged once and skipped, so a single unmapped crop
+    cannot break discovery for every other crop.
+    """
+    mapping: dict[str, list[int]] = {}
+    unmapped: list[str] = []
+    for row in fetch_all(conn, _COLTURE_SQL):
+        italian = (row.colture_name or "").strip()
+        if not italian:
+            continue
+        try:
+            english = crop_vocab.to_english(italian, kind="crop")
+        except PipelineError:
+            unmapped.append(italian)
+            continue
+        mapping.setdefault(english, []).append(int(row.colture_id))
+
+    if unmapped:
+        logger.warning(
+            "%d colture name(s) have no crop vocabulary mapping and will be "
+            "ignored by the worklist: %s",
+            len(unmapped),
+            sorted(set(unmapped)),
+        )
+    return mapping
+
+
+def _colture_names_by_id(conn: pyodbc.Connection) -> dict[int, str]:
+    return {
+        int(row.colture_id): (row.colture_name or "").strip()
+        for row in fetch_all(conn, _COLTURE_SQL)
+    }
+
+
+def build_jobs(
+    conn: pyodbc.Connection,
+    crops: tuple[CropPlan, ...],
+    worklist: WorklistSettings,
+    crop_vocab: Vocabulary,
+) -> list[Job]:
+    """Discover every (field, disease) job implied by `crops` and `worklist`."""
+    crop_to_colture = _load_crop_to_colture_ids(conn, crop_vocab)
+    colture_names = _colture_names_by_id(conn)
+    allowlist = set(worklist.field_allowlist)
+
+    jobs: list[Job] = []
+    for crop_plan in crops:
+        colture_ids = crop_to_colture.get(crop_plan.crop, [])
+        if not colture_ids:
+            logger.warning(
+                "No field layer maps to crop '%s' (check vocab/crops.yaml against "
+                "AI_agrosupport_an_colture); skipping its %d configured disease(s).",
+                crop_plan.crop,
+                len(crop_plan.diseases),
+            )
+            continue
+
+        placeholders = ",".join("?" for _ in colture_ids)
+        sql = _JOBS_SQL_TEMPLATE.format(placeholders=placeholders)
+
+        for disease_plan in crop_plan.diseases:
+            params = (*colture_ids, disease_plan.model_name.strip().upper())
+            rows = fetch_all(conn, sql, params)
+
+            kept = 0
+            for row in rows:
+                if allowlist and int(row.cmplay_id) not in allowlist:
+                    continue
+                if worklist.require_enabled_model and not bool(row.anmod_enabled):
+                    continue
+
+                jobs.append(
+                    Job(
+                        field_id=int(row.cmplay_id),
+                        crop_english=crop_plan.crop,
+                        crop_italian=colture_names.get(int(row.cmplay_colture), ""),
+                        model_name=disease_plan.model_name,
+                        disease_english=disease_plan.disease,
+                        organic=row.cmplay_imp == 3,
+                        station=(
+                            int(row.cmplay_station)
+                            if row.cmplay_station is not None
+                            else None
+                        ),
+                        anmod_id=int(row.anmod_id),
+                    )
+                )
+                kept += 1
+
+            logger.info(
+                "Worklist: crop=%s disease=%s (model=%s) -> %d field(s) "
+                "(%d row(s) matched before allowlist/enabled filters)",
+                crop_plan.crop,
+                disease_plan.disease,
+                disease_plan.model_name,
+                kept,
+                len(rows),
+            )
+
+    return jobs

prompts/_output_format.md

@@ -0,0 +1,27 @@
+## OUTPUT FORMAT
+
+Respond with a single raw JSON object and nothing else. No markdown code fences, no
+introduction, no commentary outside the JSON.
+
+| Key | Type | Rules |
+|---|---|---|
+| `full_advice` | string | Your complete advisory text, in Italian. |
+| `advice_summary` | string | Italian summary of `full_advice`, **at most 1010 characters**. It must state the recommendation: whether a treatment should be administered or not; if yes, for what reason, on what date and with which product; if no, for what reason. |
+| `apply_treatment` | integer | `1` if you recommend applying a product, `0` otherwise. |
+| `treatments` | array of strings | Names of the recommended product(s), exactly as written in the input payload. Empty array `[]` when you recommend no product. |
+| `dosage` | array of strings | Prescribed dose for each product in `treatments`, in the same order (for example `"1.5 l/ha"`). Empty array `[]` when you recommend no product. |
+| `apply_date` | string or null | Recommended application date as `YYYY-MM-DD`. `null` when you recommend no product. |
+
+Only recommend products that appear in the `allowed_products` list of the input payload.
+If `allowed_products` is empty, you must not prescribe any product.
+
+### Example shape
+
+{
+  "full_advice": "...",
+  "advice_summary": "...",
+  "apply_treatment": 1,
+  "treatments": ["PRODUCT NAME"],
+  "dosage": ["1.5 l/ha"],
+  "apply_date": "2026-08-14"
+}

prompts/advice/_default/system.md

@@ -0,0 +1,5 @@
+You are a decision support system for crop protection, advising on the risk of a plant disease for a specific crop. Your goal is to prevent infection while avoiding unnecessary treatments. You will be given the crop's phenology using the BBCH phases, weather data from the past five days, forecasts for the next five days, the deterministic phytopathological model's day-by-day risk phase output (for example FASE2 / FASE5, where a higher phase number generally indicates a more advanced and more urgent stage of infection risk), and a list of authorized treatment products with their active substances and label directives.
+
+STRICT RULES: 1. Base your interpretation strictly on the model's output; do not speculate beyond it. 2. If no treatment is currently necessary, do not discuss a hypothetical future treatment or dosage.
+
+NOTE FOR OPERATORS: this is the generic fallback advice prompt. It is used because no dedicated prompt exists yet under `prompts/advice/<crop>__<disease>/` for this crop-disease pair — see the pipeline README for how to add one. Consider writing a pair-specific prompt that explains this disease's own phase semantics, the way `prompts/advice/grapevine__downy_mildew/system.md` explains FASE2/FASE5 for downy mildew, before relying on this fallback in production.

prompts/advice/_default/user.md

@@ -0,0 +1 @@
+Based on the models' output, phenology phase and other provided information, elaborate farmers support for plant protection, and discuss the results of the model which simulated a primary infection or not. Considering the rainfall in the next days and the probable condition of the soil provide the farmer the best strategy about: 1. Whether to do a treatment in the next few days (consider forecast of output of the risk infection), 2. When is the best day and 3. Which is the best product, and how much I should apply. Do not make a chat, just speak in first person as a virtual agronomist.

prompts/advice/grapevine__downy_mildew/system.md

@@ -0,0 +1,7 @@
+You are a decision support system to prevent primary grapevine downy mildew (Plasmopara viticola) in "Costigliole Sant'Anna", Italy. Your super power is having a deterministic model for the risk of infection and your ultimate goal is to prevent the infection. You'll be given grapevine phenology using the BBCH phases, weather data from the past five days, and forecasts for the next five days. You will also be provided with a list of authorized treatment products, including their active substances and label directives, to be used if action is required.
+
+PREVIOUS KNOWLEDGE ON THE MODEL'S BEHAVIOR: The output of the model is definite. On the days marked as FASE5, conditions are favorable for the infection of zoospores on leaf surfaces and their penetration into the leaves through the stomata.
+FASE5 of the grapevine downy mildew phytopathological model refers to the critical primary infection phase. This process occurs following the dispersion of infective zoospores, which reach the leaf surfaces and successfully penetrate through the stomata. By establishing this initial infection in the first green tissues, Phase 5 marks the decisive moment that triggers the seasonal epidemic dynamics and subsequent epidemic attacks.
+In FASE2, the germination of Plasmopara viticola oospores involves the formation of a macrosporangium from which the infective zoospores are released, a process favored by high humidity and mild spring temperatures. The associated risk is the onset of primary downy mildew infections, especially following prolonged rainfall and in the presence of susceptible young foliage.
+ADDITIONAL NOTES: Main grapevine phenology phases are: BBCH < 53: leaf development, BBCH < 60: inflorescence emerge; BBCH < 70: flowering BBCH < 80: fruits development, BBCH < 90: ripening. You are smart, wise, nice, honest, and determined to promote Integrated Pest Management practices. ho cambiato idea, scrivilo in Italiano.
+STRICT RULES: 1. Stick only to the output of the model for any interpreting or explanation. 2. If there is no necessity for a treatment, don't discuss the probable future situation may need a treatment or the dosage.

prompts/advice/grapevine__downy_mildew/user.md

@@ -0,0 +1 @@
+Based on the models' output, phenology phase and other provided information, elaborate farmers support for plant protection, and discuss the results of the model which simulated a primary infection or not. Considering the rainfall in the next days and the probable condition of the soil provide the farmer the best strategy about: 1. Whether to do a treatment in the next few days (consider forecast of output of the risk infection), 2. When is the best day and 3. Which is the best product, and how much I should apply. Do not make a chat, just speak in first person as a virtual agronomist.

prompts/query_synthesis/_default.md

@@ -0,0 +1,74 @@
+# SYSTEM PROMPT: AGRONOMIC QUERY SYNTHESIZER
+
+## ROLE AND PURPOSE
+You are an expert agronomic query synthesizer for a vector search system. Your task is to transform structured JSON containing crop information, disease threats, phenological growth stages (BBCH codes), and multi-day weather forecasts (up to 6 days) into semantically rich natural language search queries.
+
+The output query text will be embedded to retrieve chemical treatments (e.g., fungicides, bactericides) from a vector database.
+
+---
+
+## CRITICAL CONSTRAINTS & RULES
+
+### 1. Context Usage vs. Anonymization (STRICT)
+* **Full Context Utilization:** Use the `"crop"` and `"disease"` fields strictly to infer biological pathogen mechanisms (e.g., spore germination styles, incubation speeds, Oomycete vs. Ascomycete behavior) and crop tissue sensitivity (e.g., vulnerability of pre-flowering structures).
+* **NO EXPLICIT NAMES:** You MUST NOT mention the specific names of the crop (e.g., "grapevine", "Vitis vinifera") or disease (e.g., "peronospora", "downy mildew", "Plasmopara viticola") anywhere in the generated output text. Metadata filtering handles crop and disease downstream; naming them in the vector query degrades vector similarity scoring.
+
+### 2. Multi-Condition Weather Analysis & Query Splitting
+Analyze weather variations across the 1–6 day forecast window:
+* **Single Query Condition (Uniform Weather):** If weather metrics across all days are consistent (e.g., all dry or all persistently wet), generate **one** synthesized search query covering the entire window.
+* **Dual Query Condition (Drastic Weather Shift):** If weather conditions vary significantly across the period (e.g., clear/mild baseline days transitioning into heavy rainfall and prolonged leaf wetness), generate **two separate queries**:
+  * **QUERY 1 (Baseline / Dry Window):** Focused on preventive, protective surface coverage and residual control during mild conditions.
+  * **QUERY 2 (High-Risk / Wet Window):** Focused on high rainfastness, wash-off resistance, rapid uptake, systemic/translaminar mobility, and curative/incubation-stopping activity during heavy rain or high leaf wetness.
+
+---
+
+## PHENOLOGY & METEOROLOGICAL MAPPING
+
+* **Phenological Phase (BBCH):** Translate stage into plant structure vulnerabilities (e.g., BBCH 53 = visible inflorescences requiring non-phytotoxic, gentle preventive protection for delicate flowering structures).
+* **Rainfall & Leaf Wetness:**
+  * Low rain / low wetness: Surface contact, preventive barrier, long residual protection.
+  * Heavy rain (>5–10 mm) / High wetness (>300 min) / RH (>80%): Severe wash-off risk and peak spore germination. Demands rainfastness after drying, translaminar/systemic redistribution, and early curative action.
+
+---
+
+## OUTPUT FORMATTING INSTRUCTIONS
+* Output your response **ONLY** as a valid JSON string list (e.g., `["QUERY1"]` or `["QUERY1", "QUERY2"]`).
+* If weather is uniform: Return a list containing a single string query: `["QUERY1"]`.
+* If weather shifts drastically: Return a list containing two string queries: `["QUERY1", "QUERY2"]`.
+* Do NOT include markdown code fences (such as ```json or ```), introductory text, greetings, headers, or explanations. Return strictly the raw JSON array string.
+
+---
+
+## FEW-SHOT EXAMPLE
+
+### INPUT JSON:
+{
+  "crop": "grapevine",
+  "disease": "peronospora",
+  "phenology_phase": "BBCH 53 - Infiorescenze visibili",
+  "weather_forecasts": [
+    {
+      "date": "26-07-2026",
+      "min_air_temperature_c": 9.6, "max_air_temperature_c": 23.9, "average_air_temperature_c": 16.2,
+      "min_relative_humidity_percent": 25, "max_relative_humidity_percent": 85, "average_relative_humidity_percent": 53,
+      "rainfall_mm": 0.0, "leaf_wetness_minutes": 300, "potential_evapotranspiration_mm": 3.6
+    },
+    {
+      "date": "27-07-2026",
+      "min_air_temperature_c": 10.6, "max_air_temperature_c": 21.3, "average_air_temperature_c": 15.8,
+      "min_relative_humidity_percent": 39, "max_relative_humidity_percent": 92, "average_relative_humidity_percent": 58,
+      "rainfall_mm": 0.8, "leaf_wetness_minutes": 720, "potential_evapotranspiration_mm": 2.9
+    },
+    {
+      "date": "28-07-2026",
+      "min_air_temperature_c": 10.6, "max_air_temperature_c": 15.6, "average_air_temperature_c": 12.9,
+      "min_relative_humidity_percent": 76, "max_relative_humidity_percent": 99, "average_relative_humidity_percent": 94,
+      "rainfall_mm": 17.4, "leaf_wetness_minutes": 300, "potential_evapotranspiration_mm": 4.0
+    }
+  ]
+}
+
+### OUTPUT:
+QUERY 1: Preventive protective treatment suitable for pre-flowering stage with visible inflorescences (BBCH 53) during dry to mild conditions with moderate humidity and low wash-off risk. The product must provide a durable protective barrier over delicate emerging floral structures to inhibit spore germination prior to high-moisture infection events.
+
+QUERY 2: High-performance systemic or translaminar fungicide for pre-flowering stage with visible inflorescences (BBCH 53) exposed to severe infection pressure, characterized by extreme relative humidity up to 99%, prolonged leaf wetness up to 720 minutes, and heavy rainfall reaching 17.4 mm. The treatment must feature rapid plant absorption, excellent rainfastness after drying, wash-off resistance, and early curative capability to halt fungal incubation during extended wet periods.

vocab/crops.yaml

@@ -0,0 +1,421 @@
+# Canonical English crop names and Italian → English normalisation map.
+#
+# normalize: Italian term (lowercase) → canonical English.
+
+canonical_values:
+  - grapevine
+  - apple
+  - pear
+  - quince
+  - medlar
+  - potato
+  - tomato
+  - cucumber
+  - gherkin
+  - melon
+  - watermelon
+  - lettuce
+  - spinach
+  - herbs_fresh
+  - broccoli
+  - cauliflower
+  - cabbage
+  - savoy_cabbage
+  - artichoke
+  - asparagus
+  - basil
+  - bay_laurel
+  - bergamot
+  - onion
+  - garlic
+  - shallot
+  - orange
+  - lemon
+  - lemon_balm
+  - mandarin
+  - clementine
+  - lime
+  - grapefruit
+  - peach
+  - nectarine
+  - apricot
+  - cherry
+  - plum
+  - almond
+  - walnut
+  - hazelnut
+  - chestnut
+  - pistachio
+  - olive
+  - oregano
+  - parsley
+  - persimmon
+  - pomegranate
+  - pomelo
+  - pome_fruit
+  - stone_fruit
+  - pumpkin
+  - rosemary
+  - sage
+  - spring_onion
+  - strawberry
+  - wheat
+  - barley
+  - maize
+  - marjoram
+  - mint
+  - sunflower
+  - soybean
+  - kiwi
+  - pepper
+  - eggplant
+  - zucchini
+  - celery
+  - carrot
+  - cardoon
+  - chervil
+  - chives
+  - cypress
+  - leek
+  - bean
+  - green_bean
+  - pea
+  - snow_pea
+  - lentil
+  - rapeseed
+  - sugar_beet
+  - beetroot
+  - tobacco
+  - rose
+  - ornamental_plants
+  - ornamental_trees
+  - forest_trees
+  - nut_trees
+  - fennel
+  - radish
+  - rocket
+  - turnip
+  - chicory
+  - endive
+  - escarole
+  - citrus
+  - turf
+  - chard
+  - fresh_legumes_with_pod
+  - avocado
+  - bitter_orange
+  - blackberry
+  - blueberry
+  - cedar
+  - currant
+  - edible_flowers
+  - elderberry
+  - gooseberry
+  - hawthorn
+  - june_mustard
+  - pineapple
+  - raspberry
+  - tarragon
+  - thyme
+  - valerianella
+  - watercress
+  - vegetables
+  - sweet_potato
+  - swiss_chard
+  - parsnip
+  - rutabaga
+  - dandelion
+  - citron
+  - kohlrabi
+  - red_raspberry
+  - black_raspberry
+  - yellow_raspberry
+
+normalize:
+  # Grapevine
+  vite: grapevine
+  "vite (uva da vino)": grapevine
+  "vite (uva da tavola)": grapevine
+  "uva da vino": grapevine
+  "uva da tavola": grapevine
+  vigna: grapevine
+  grape: grapevine
+
+  # Apple / pear / quince / medlar
+  melo: apple
+  pero: pear
+  cotogno: quince
+  nespolo: medlar
+
+  # Potato
+  patata: potato
+  "patata (in campo)": potato
+
+  # Tomato
+  pomodoro: tomato
+  "pomodoro (in campo)": tomato
+  "pomodoro (in serra)": tomato
+
+  # Cucumber
+  cetriolo: cucumber
+  "cetriolo (in campo)": cucumber
+  "cetriolo (in serra)": cucumber
+
+  # Melon / watermelon
+  melone: melon
+  cocomero: watermelon
+  anguria: watermelon
+  "melone (in campo)": melon
+  "melone (in serra)": melon
+  "cocomero (in campo)": watermelon
+  "cocomero (in serra)": watermelon
+
+  # Lettuce / salad greens
+  lattuga: lettuce
+  "lattughe e insalate": lettuce
+  insalata: lettuce
+  salad: lettuce
+
+  # Spinach / similar
+  spinaci: spinach
+  "spinaci e simili": spinach
+  "spinach and similar": spinach
+
+  # Fresh herbs
+  "erbe fresche": herbs_fresh
+  "fresh herbs": herbs_fresh
+
+  # Brassicas
+  broccolo: broccoli
+  cavolfiore: cauliflower
+  "cavolo cappuccio": cabbage
+  "cavolo verza": savoy_cabbage
+  cavolo: cabbage
+  "savoy cabbage": savoy_cabbage
+
+  # Onion
+  cipolla: onion
+
+  # Citrus
+  arancio: orange
+  limone: lemon
+  mandarino: mandarin
+  clementino: clementine
+  lime: lime
+  pompelmo: grapefruit
+
+  # Cereals
+  frumento: wheat
+  grano: wheat
+  orzo: barley
+  mais: maize
+  granoturco: maize
+
+  # Others
+  girasole: sunflower
+  soia: soybean
+
+  # Stone fruit
+  pesco: peach
+  nettarina: nectarine
+  nettarine: nectarine
+  albicocco: apricot
+  ciliegio: cherry
+  susino: plum
+
+  # Nut trees
+  mandorlo: almond
+  noce: walnut
+  nocciolo: hazelnut
+  castagno: chestnut
+  pistacchio: pistachio
+
+  # Olive
+  olivo: olive
+
+  fragola: strawberry
+
+  # Vegetables
+  carciofo: artichoke
+  scalogno: shallot
+  fagiolino: green_bean
+  fagiolo: bean
+  "pisello mangiatutto": snow_pea
+  lenticchia: lentil
+  cetriolino: gherkin
+
+  # Ornamentals / forest
+  "ornamentali arboree": ornamental_trees
+  "floreali e ornamentali": ornamental_plants
+  "floreali, ornamentali e forestali": ornamental_plants
+  forestali: forest_trees
+  kiwi: kiwi
+  peperone: pepper
+  melanzana: eggplant
+  zucchina: zucchini
+  sedano: celery
+  carota: carrot
+  aglio: garlic
+  porro: leek
+  pisello: pea
+  colza: rapeseed
+  "barbabietola da zucchero": sugar_beet
+  tabacco: tobacco
+  rosa: rose
+  "piante ornamentali": ornamental_plants
+
+  # Herbs and aromatics
+  asparago: asparagus
+  basilico: basil
+  alloro: bay_laurel
+  bergamotto: bergamot
+  cardo: cardoon
+  cerfoglio: chervil
+  "erba cipollina": chives
+  cipollotto: spring_onion
+  cipollino: spring_onion
+  melissa: lemon_balm
+  maggiorana: marjoram
+  menta: mint
+  origano: oregano
+  prezzemolo: parsley
+  rosmarino: rosemary
+  salvia: sage
+
+  # Other crops
+  cipresso: cypress
+  zucca: pumpkin
+  squash: pumpkin
+  cachi: persimmon
+  melograno: pomegranate
+  pomelo: pomelo
+
+  # Vegetables (additional)
+  finocchio: fennel
+  ravanello: radish
+  rucola: rocket
+  rughetta: rocket
+  rapa: turnip
+  navone: rutabaga
+  cicoria: chicory
+  indivia: endive
+  "indivia belga": endive
+  "indivia scarola": escarole
+  scarola: escarole
+  "barbabietola rossa": beetroot
+  "barbabietola da orto": beetroot
+  barbabietola: beetroot
+
+  # Citrus (generic – when label uses group name without individual expansion)
+  agrumi: citrus
+  citrus: citrus
+
+  # Turf / grass
+  "tappeti erbosi": turf
+  "prati ornamentali": turf
+  "campi da golf": turf
+  "campi sportivi": turf
+  turf: turf
+
+  # Leaf beet / chard
+  chard: chard
+  "bietola da foglia": chard
+  "bietola da costa": swiss_chard
+  bietola: chard
+  swiss_chard: swiss_chard
+
+  # Fresh legumes with pod (bean, pea, etc.)
+  fresh_legumes_with_pod: fresh_legumes_with_pod
+  "legumi freschi con baccello": fresh_legumes_with_pod
+  "legumi freschi (con baccello)": fresh_legumes_with_pod
+  "legumi freschi": fresh_legumes_with_pod
+
+  # Tropical / subtropical fruit
+  avocado: avocado
+  ananas: pineapple
+  pineapple: pineapple
+
+  # Citrus
+  "arancio amaro": bitter_orange
+  bitter_orange: bitter_orange
+  cedro: citron
+  citron: citron
+  "cedro del libano": cedar
+  cedar: cedar
+
+  # Berries and small fruit
+  mora: blackberry
+  blackberry: blackberry
+  mirtillo: blueberry
+  blueberry: blueberry
+  lampone: raspberry
+  lamponi: raspberry
+  red_raspberry: red_raspberry
+  black_raspberry: black_raspberry
+  yellow_raspberry: yellow_raspberry
+  "lampone rosso": red_raspberry
+  "lampone nero": black_raspberry
+  "lampone giallo": yellow_raspberry
+  raspberry: raspberry
+  ribes: currant
+  "ribes nero": currant
+  "ribes rosso": currant
+  currant: currant
+  "uva spina": gooseberry
+  gooseberry: gooseberry
+  sambuco: elderberry
+  elderberry: elderberry
+
+  # Salad greens and leafy vegetables
+  valerianella: valerianella
+  songino: valerianella
+  gallinella: valerianella
+  dolcetta: valerianella
+  crescione: watercress
+  watercress: watercress
+
+  # Brassicas / mustard family
+  "senape juncea": june_mustard
+  "senape estiva": june_mustard
+  june_mustard: june_mustard
+
+  # Herbs
+  dragoncello: tarragon
+  estragon: tarragon
+  tarragon: tarragon
+  timo: thyme
+  thyme: thyme
+
+  # Ornamentals / specialty
+  "fiori eduli": edible_flowers
+  edible_flowers: edible_flowers
+  azzeruolo: hawthorn
+  biancospino: hawthorn
+  hawthorn: hawthorn
+
+  # Root vegetables
+  "patata americana": sweet_potato
+  batata: sweet_potato
+  sweet_potato: sweet_potato
+  pastinaca: parsnip
+  parsnip: parsnip
+  "rapa svizzera": rutabaga
+  rutabaga: rutabaga
+  "cavolo rapa": kohlrabi
+  kohlrabi: kohlrabi
+
+  # Generic crop groups
+  pomacee: pome_fruit
+  drupacee: stone_fruit
+  "frutta a guscio": nut_trees
+  "frutta a nocciolo": stone_fruit
+  vegetables: vegetables
+  ortaggi: vegetables
+  orticole: vegetables
+  verdure: vegetables
+  "ortaggi in foglia": vegetables
+
+  # Weeds / broadleaf targets
+  tarassaco: dandelion
+  "dente di leone": dandelion
+  dandelion: dandelion

vocab/diseases.yaml

@@ -0,0 +1,290 @@
+# Canonical English disease names and Italian → English normalisation map.
+#
+# normalize: Italian term (lowercase) → canonical English.
+
+canonical_values:
+  - downy mildew
+  - powdery mildew
+  - scab
+  - brown spot
+  - brown rot
+  - black rot
+  - phomopsis (dead-arm)
+  - red fire disease
+  - alternaria
+  - botrytis (grey mould)
+  - anthracnose
+  - phytophthora
+  - pythium
+  - fusarium
+  - rust
+  - white rust
+  - leaf spot
+  - cercospora leaf spot
+  - bacterial disease
+  - bacteriosis
+  - fire blight
+  - crown gall
+  - clubroot
+  - late blight
+  - early blight
+  - damping-off
+  - white mould
+  - verticillium wilt
+  - armillaria root rot
+  - esca (grapevine trunk disease)
+  - eutypa dieback
+  - black dead arm
+  - penicillium mould
+  - septoria leaf blotch
+  - septoria leaf spot
+  - net blotch
+  - rhynchosporium
+  - smut
+  - bunts
+  - leaf curl
+  - peach leaf curl
+  - shot hole
+  - shot hole disease
+  - coryneum blight
+  - gummosis
+  - olive peacock spot
+  - olive leaf spot
+  - olive knot
+  - sclerotinia
+  - canker
+  - european canker
+  - cypress canker
+  - pistachio branch canker
+  - walnut blight
+  - cylindrosporium leaf spot
+  - cherry leaf spot
+  - chestnut leaf spot
+  - cytospora canker
+  - common leaf spot (strawberry)
+  - leaf scorch (strawberry)
+  - sooty mould
+  - cladosporium
+  - mal secco
+  - fungal diseases
+  - insect pests
+
+normalize:
+  # Downy mildew (peronospora) – multiple pathogens
+  peronospora: downy mildew
+  "peronospora (plasmopara viticola)": downy mildew
+  "peronospora (peronospora destructor)": downy mildew
+  "peronospora (hyaloperonospora brassicae)": downy mildew
+  "peronospora (pseudoperonospora cubensis)": downy mildew
+  "peronospora (bremia lactucae)": downy mildew
+  downy_mildew: downy mildew
+
+  # Powdery mildew
+  oidio: powdery mildew
+  "patina bianca": powdery mildew
+  "patina bianca (tilletiopsis spp.)": powdery mildew
+  "mal bianco": powdery mildew
+  oidium: powdery mildew
+
+  # Scab
+  ticchiolatura: scab
+  "ticchiolatura (venturia inaequalis)": scab
+  "ticchiolatura (venturia pirina)": scab
+
+  # Brown spot / stemphylium
+  "maculatura bruna": brown spot
+  "maculatura bruna (stemphylium vesicarium)": brown spot
+
+  # Brown rot (Monilinia)
+  "marciume bruno": brown rot
+  "marciume bruna": brown rot
+  brown_rot: brown rot
+  moniliosi: brown rot
+  monilinosi: brown rot
+  monilia: brown rot
+
+  # Black rot / guignardia
+  "marciume nero": black rot
+  "black-rot": black rot
+  "black rot": black rot
+  "marciume nero (guignardia bidwellii)": black rot
+
+  # Phomopsis / excoriosis
+  escoriosi: "phomopsis (dead-arm)"
+  "escoriosi (phomopsis viticola)": "phomopsis (dead-arm)"
+  "escoriosi della vite": "phomopsis (dead-arm)"
+
+  # Red fire disease (grapevine)
+  "rossore parassitario": red fire disease
+  red_fire_disease: red fire disease
+
+  # Alternaria
+  alternariosi: alternaria
+  "alternariosi (alternaria spp.)": alternaria
+
+  # Botrytis / grey mould
+  botrite: "botrytis (grey mould)"
+  botrytis: "botrytis (grey mould)"
+  "muffa grigia": "botrytis (grey mould)"
+  "marciume grigio": "botrytis (grey mould)"
+
+  # Anthracnose / glomerella
+  glomerella: anthracnose
+  "glomerella (colletotrichum spp.)": anthracnose
+  antracnosi: anthracnose
+
+  # Phytophthora (non-oomycete uses)
+  "allupatura": phytophthora
+  "allupatura (phytophthora sp.)": phytophthora
+  phytophthora: phytophthora
+  allupatura: phytophthora
+
+  # Rust
+  ruggine: rust
+  "ruggine (stemphylium spp.)": rust
+  "ruggine bianca": white rust
+  white_rust: white rust
+
+  # Late blight (Phytophthora infestans – tomato, potato)
+  "peronospora (blight)": late blight
+  "peronospora (phytophthora infestans)": late blight
+  "peronospora (phytophtora infestans)": late blight
+  late_blight: late blight
+
+  # Fusarium
+  fusariosi: fusarium
+
+  # Pythium
+  "pythium blight": pythium
+  pythium_blight: pythium
+
+  # Bacterial diseases (generic term; specific species go into extraction_notes)
+  batteriosi: bacteriosis
+  "batteriosi (pseudomonas spp.)": bacteriosis
+  "batteriosi (xanthomonas spp.)": bacteriosis
+  "batteriosi (erwinia spp.)": bacteriosis
+  "piticchia batterica": bacteriosis
+  "piticchia batterica (pseudomonas syringae)": bacteriosis
+  "mal secco del noce": walnut blight
+  xanthomonas_arboricola_pv_juglandis: walnut blight
+
+  # Fire blight
+  fire_blight: fire blight
+
+  # Leaf curl (Taphrina / bolla)
+  bolla: "leaf curl"
+  "bolla (taphrina spp.)": "leaf curl"
+  "bolla (taphrina deformans)": "leaf curl"
+  leaf_curl: leaf curl
+  peach_leaf_curl: peach leaf curl
+
+  # Shot hole / coryneum blight
+  corineo: "coryneum blight"
+  "corineo (coryneum spp.)": "coryneum blight"
+  coryneum_blight: coryneum blight
+  vaiolatura: "shot hole"
+  "vaiolatura (stigmina carpophila)": "shot hole"
+  shot_hole: shot hole
+  shot_hole_disease: shot hole disease
+
+  # Gummosis (Phytophthora on citrus)
+  gommosi: gummosis
+  "gommosi (phytophthora spp.)": gummosis
+
+  # Olive peacock spot
+  "occhio di pavone": "olive peacock spot"
+  "occhio di pavone (spilocea oleaginea)": "olive peacock spot"
+  olive_peacock_spot: olive peacock spot
+  olive_leaf_spot: olive leaf spot
+
+  # Olive knot
+  "rogna dell'olivo": olive knot
+  "tubercolosi dell'olivo": olive knot
+  olive_knot: olive knot
+
+  # Lebbra / olive anthracnose
+  lebbra: anthracnose
+  "lebbra (gloeosporium olivarum)": anthracnose
+  "lebbra (colletotrichum gloeosporioides)": anthracnose
+
+  # Sclerotinia
+  sclerotinia: sclerotinia
+  "sclerotinia (sclerotinia spp.)": sclerotinia
+
+  # Cankers
+  "cancri rameali": canker
+  "seccume rameale": "cytospora canker"
+  "seccume rameale (cytospora leucostoma)": "cytospora canker"
+  "mal dello stacco del nocciolo (cytospora spp.)": "cytospora canker"
+  cytospora_canker: cytospora canker
+  european_canker: european canker
+  "cancro del cipresso": cypress canker
+  cypress_canker: cypress canker
+  "cancro rameale del pistacchio": pistachio branch canker
+  botryosphaeria_dothidea: pistachio branch canker
+
+  # Cylindrosporium (cherry leaf spot)
+  cilindrosporiosi: "cylindrosporium leaf spot"
+  "cilindrosporiosi del ciliegio": "cylindrosporium leaf spot"
+  "cilindrosporiosi del ciliegio (blumeriella japii)": "cylindrosporium leaf spot"
+  cylindrosporium_leaf_spot: cylindrosporium leaf spot
+  cherry_leaf_spot: cherry leaf spot
+
+  # Cercospora leaf spot
+  cercospora_leaf_spot: cercospora leaf spot
+
+  # Fersa del castagno
+  "fersa del castagno": chestnut leaf spot
+  "fersa del castagno (mycosphaerella spp.)": chestnut leaf spot
+  chestnut_leaf_spot: chestnut leaf spot
+
+  # Septoria
+  septoriosi: "septoria leaf blotch"
+  "septoriosi (septoria spp.)": "septoria leaf blotch"
+  septoria_leaf_blotch: septoria leaf blotch
+  septoria_leaf_spot: septoria leaf spot
+
+  # Strawberry leaf diseases
+  "maculatura comune (fragola)": common leaf spot (strawberry)
+  common_leaf_spot_(strawberry): common leaf spot (strawberry)
+  "scorch fogliare (fragola)": leaf scorch (strawberry)
+  leaf_scorch_(strawberry): leaf scorch (strawberry)
+
+  # Mal del piede
+  "mal del piede": fusarium
+
+  # Generic leaf spot (underscore alias)
+  leaf_spot: leaf spot
+
+  # Marciume (Monilia, Nectria, Ascochyta)
+  "marciume (monilia spp.)": brown rot
+  "marciume (nectria spp.)": canker
+  "marciume (ascochyta spp.)": leaf spot
+
+  # Sooty mould (fumaggini)
+  fumaggini: sooty mould
+  sooty_mould: sooty mould
+
+  # Cladosporium (tomato leaf mold / cladosporiosi)
+  cladosporiosi: cladosporium
+  "cladosporiosi (cladosporium fulvum)": cladosporium
+  "cladosporiosi (passalora fulva)": cladosporium
+  "cladosporium fulvum": cladosporium
+
+  # Mal secco (citrus – Phoma/Deuterophoma tracheiphila)
+  "mal secco": mal secco
+  mal_secco: mal secco
+  "mal secco (deuterophoma tracheiphila)": mal secco
+  "mal secco (phoma tracheiphila)": mal secco
+
+  # Generic target groups (adjuvants / tank-mix partners – no specific pathogen on label)
+  fungal_diseases: fungal diseases
+  "malattie fungine": fungal diseases
+  "patogeni fungini": fungal diseases
+  fungicidi: fungal diseases
+  insetticidi: insect pests
+  insect_pests: insect pests
+  fitofagi: insect pests
+  insetti: insect pests
+  "parassiti animali": insect pests
+  acaricidi: insect pests