# Fix Fallback Reasonableness and Trust Indicators

## Overview
The goal of this task was to address an issue where the "Used - 365 days avg." fallback mechanism (introduced to support sparse data ingestion) was generating overly optimistic profit estimates. These estimates bypassed the standard "reasonableness checks" (like the Amazon price ceiling), leading to inflated listing prices (e.g., $4,000 for a common book) being accepted without scrutiny.

Additionally, the user requested that the "List at" price display be cleaned up by removing the visual `(Est.)` tag, while still retaining a reduced trust indicator in the "Profit Confidence" score.

## Challenges
1.  **Bypassed Validation Logic:** The `analyze_sales_performance` function in `keepa_deals/stable_calculations.py` had an early return statement in the fallback logic block. This caused deals using the fallback price to skip the subsequent "Amazon Ceiling Logic" and "XAI Reasonableness Check", which are critical for filtering out unrealistic prices.
2.  **Balancing Trust vs. UI Clutter:** The user wanted to signal lower confidence for estimated prices without cluttering the main price column with text tags like `(Est.)`.
3.  **Database State:** The existing database contained "optimistic" deals generated by the flawed logic, necessitating a clean reset to restore data integrity.

## Actions Taken

### 1. Refactored `analyze_sales_performance` (Reasonableness Check Enforcement)
-   **File:** `keepa_deals/stable_calculations.py`
-   **Change:** Restructured the function to remove the early return in the sparse data fallback block.
-   **Outcome:** Now, when a fallback price (e.g., `stats.avg365`) is used, it flows through the exact same validation pipeline as a standard "Inferred Sale" price:
    *   **Amazon Ceiling:** The price is capped at 90% of the lowest Amazon New price (Current, 180d, or 365d avg).
    *   **XAI Check:** The capped price is sent to the AI (Grok) to verify if it is reasonable given the product's metadata (Title, Category, Rank).
-   **Result:** Prices that are statistically high but market-unrealistic are now either capped or rejected.

### 2. Adjusted Trust Indicators (UI Cleanup)
-   **File:** `keepa_deals/processing.py`
-   **Change:** Removed the code that appended the string `(Est.)` to the `List at` field.
-   **Change:** Retained the logic that forces the `Profit Confidence` score to **"Low (Est.)"** when `price_source == 'Keepa Stats Fallback'`.
-   **Outcome:** The "List at" column remains clean (e.g., "$45.00"), but the user is still warned via the low confidence score that the price is an estimate.

### 3. Verified Sparse Data Logic
-   **File:** `keepa_deals/stable_calculations.py` (`infer_sale_events`)
-   **Verification:** Confirmed that the "Sparse Data" logic is active and correctly implements a **30-day lookahead** for rank drops. This allows the system to infer sales even when data points are sparse, by detecting rank improvements across wider time gaps.

### 4. Database Reset
-   **Script:** `Diagnostics/reset_database.py`
-   **Action:** Executed the script to drop and recreate the `deals` and `user_restrictions` tables.
-   **Outcome:** All previous deals (including the overly optimistic ones) were cleared, ensuring the dashboard will repopulate with clean, validated data.

## Outcome
**SUCCESS.**
The system now correctly validates fallback prices against market realities (Amazon Ceiling) and AI logic. The UI presentation meets the user's preference for cleanliness while maintaining transparency about data confidence. The database has been reset to provide a clean slate for the improved logic.