# Feature Documentation: Deals, Dashboard & Data Sourcing This document details the functionality, logic, and specifications for the Deals Dashboard, Query Configuration, and Data Sourcing features. --- ## 1. Deals Dashboard **Route:** `/dashboard` **Template:** `templates/dashboard.html` **API Endpoint:** `/api/deals` (`wsgi_handler.py`) ### Overview The Dashboard is the central hub for viewing and analyzing arbitrage opportunities. It presents a grid of deals sourced from Keepa, enriched with calculated metrics (Profit, ROI, Sales Rank), and allows for powerful filtering and sorting. ### Key Features * **Data Grid:** Displays deals in a responsive table. Columns are defined in `Documentation/Dashboard_Specification.md`. * **Self-Aware Tooltips (March 2026):** Hovering over any table header or filter label instantly displays a contextual tooltip explaining its purpose or underlying data logic. * These tooltips are generated by the AI (Grok) which reads `Documentation/` files as its source of truth. * The results are stored persistently in `tooltip_cache.json` on the backend to ensure instant loading without waiting for an API response. * *UI Note:* To preserve the strict 1200px table width limit, there are no `?` icons; the text labels themselves are the triggers (`cursor: help`). * **Filtering:** Users can filter by Keyword, Max Sales Rank, Minimum Profit, Margin, Deal Trust, Seller Trust, and Price Drops. * **New Checkboxes:** "Hide Gated" (excludes restricted items) and "Hide AMZ Offers" (excludes items sold by Amazon). * *Logic:* Filters are applied server-side by the `/api/deals` endpoint SQL query. * **"Any" Logic:** Setting a filter to 0 ("Any") excludes it from the query, ensuring that NULL/Negative values are not hidden by default. * **Sorting:** Columns like "Profit", "Rank", "Update Time" are sortable. * **Real-time Updates (The "Janitor"):** * **"Refresh Deals" Button:** Manually reloads the grid to show the latest data. **Note:** As of Jan 2026, this button does **not** trigger the "Janitor" cleanup task (which runs automatically every 4 hours) to prevent accidental data loss. * **Passive Notification:** The dashboard polls `/api/deal-count` (filtered) every 30 seconds. It compares this count against the local filtered record count. If the server count differs, a notification ("New Deals Available") appears. * **Recalculation:** A "Recalculate" feature allows updating business metrics (Profit, ROI) based on changed settings (Tax, Prep Fee) without re-fetching data from Keepa. ### Gated Column States The "Gated" column indicates the user's restriction status on Amazon: * **Spinner:** Check is pending/queued. * **Green Check:** Approved / Not Restricted. * **Red X:** Restricted. (Clicking opens "Apply to Sell"). * **Broken Icon (⚠):** API Error (e.g., Timeout, 403 Forbidden). Hovering shows "API Error". ### My Mentor (AI Overlay) When expanding a deal's details: * A "My Mentor" section appears prominently at the top of the overlay. * **Choose Your Mentor:** Users can switch between 4 AI personas to get different perspectives: * **CFO:** Risk-averse, profit-focused. * **Flipper:** Speed/Volume focused. * **Professor:** Educational, explains concepts. * **Quant:** Statistical, data-heavy. * It asynchronously fetches an AI analysis (`/api/ava-advice/`) powered by `grok-4-fast-reasoning`. * Provides 50-80 words of actionable advice based on the deal's specific metrics. * **Formatting:** The system explicitly prompts the AI to return **HTML** (e.g., ``, `
`) instead of Markdown. The frontend renders this safely using `innerHTML`. ### Mentor Chat Integration A persistent, client-side chat interface accessible via the "Mentor" navigation item. * **Visual:** Fixed overlay (505x540px) at the bottom-right of the screen. * **Shared State:** The active persona (e.g., "Olyvia") is synchronized between this Chat Window and the Deal Details "My Mentor" overlay using `localStorage`. Changing one updates the other instantly. * **Behavior:** Features a "Click to Submit" model (Enter key disabled) to prevent accidental sends. * **Rendering:** Mentor responses are rendered as trusted HTML (supporting bold/line breaks), while user input is strictly treated as text to prevent XSS. --- ## 2. Deals Query Configuration (Admin Only) **Access Control:** This feature is strictly restricted to **Admin** users. Regular users cannot access this page. **Route:** `/deals` **Template:** `templates/deals.html` **Storage:** `keepa_query.json` ### Overview This page acts as a configuration interface for the Keepa API query used during the "Data Sourcing" (Backfill) process. Note: Previous admin functions like the "Backfill Limiter," "Manual Data Refresh," and "Danger Zone" database reset were removed in Feb 2026 to prevent accidental data loss and system instability. ### Functionality * **JSON Editor:** Provides a text area to edit the raw JSON query object. * **Validation:** On submission (`POST /deals`), the backend attempts to parse the JSON. If invalid, an error is flashed, and the file is not saved. * **Persistence:** Valid queries are saved to `keepa_query.json` in the root directory. This file is read by `trigger_backfill_task.py` to determine which products to fetch from Keepa. * **Standards:** The system now supports and recommends `dateRange: 4` (All Combined) to ensure the maximum 3-year history is available for AI analysis. This **must** be paired with `sortType: 4` (Last Update) to ensure data freshness. --- ## 3. Data Sourcing (Hidden) **Note:** The explicit "Data Sourcing" page has been removed from the navigation menu for all users as of Phase 1 Access Control updates. Backfill operations are now primarily managed via CLI or automatic scheduled tasks, though the endpoints and scripts remain functional in the backend. ### Architecture Note The backfill process interacts directly with the `system_state` table in the database to track the "Backfill Page" and "Watermark", ensuring that scans can resume if interrupted.