Design

How it works

1. SuiviBourse loads your portfolio from either a config.yaml file (manual mode) or from CSV/XLSX transaction files (events mode).
2. Three independent scheduled jobs run in parallel:
   • Scraping — fetches current prices from Yahoo! Finance.
   • Ingestion — reloads and re-aggregates your portfolio configuration.
   • Backfill — progressively fills in historical prices (events mode only).
3. Every data point is written to InfluxDB 3 Core in the portfolio_metrics measurement.
4. Grafana reads InfluxDB (SQL datasource) to display your portfolio in a ready-made dashboard.
5. A legacy Prometheus /metrics endpoint is also exposed for backward compatibility with pre-v4 deployments.

Architecture

                             SuiviBourse (app)
┌──────────────────────┬──────────────────────┬──────────────────────┐
│      SCRAPING        │      INGESTION       │      BACKFILL        │
│    (every 120s)      │    (every 300s)      │    (every 60s)       │
│                      │                      │  events mode only    │
│ • yfinance.Ticker()  │ • Load config or     │ • Find first BUY     │
│ • Current price,     │   event files        │ • Check oldest point │
│   volume, P/E, …     │ • Validate           │   in InfluxDB        │
│ • Write InfluxDB     │ • Aggregate          │ • Fetch 1 chunk of   │
│ • Update legacy      │ • Cache (by mtime)   │   history (≈1 year)  │
│   Prometheus gauges  │ • Update shares[]    │ • Enrich + write     │
└──────────┬───────────┴──────────┬───────────┴──────────┬───────────┘
           │                      │                       │
           └──────────────────────┼───────────────────────┘
                                  ▼
                        ┌───────────────────┐        ┌──────────────────┐
                        │  InfluxDB 3 Core  │◄──────►│     Grafana      │
                        │ portfolio_metrics │  SQL   │    dashboard     │
                        └───────────────────┘        └──────────────────┘
                                  ▲
                                  │ (legacy, snapshot only)
                        ┌───────────────────┐
                        │ Prometheus /metrics│  :8081
                        └───────────────────┘

Three independent schedules

The application runs three separate scheduled jobs that operate independently. Because they are decoupled, an error in one never blocks the others.

Job	Default interval	Env variable	Purpose	Error handling
Scraping	120 s	SB_SCRAPING_INTERVAL	Fetch live prices from Yahoo! Finance	Retries with exponential backoff on rate limits
Ingestion	300 s	SB_INGESTION_INTERVAL	Reload & re-aggregate the portfolio	Keeps the previous valid configuration
Backfill	60 s	SB_BACKFILL_INTERVAL	Fill historical price data (events mode)	Retries the same chunk on the next cycle

Why three separate schedules?

• Isolation — if your event files contain an error, price scraping keeps running with the last valid configuration.
• Different frequencies — stock prices change constantly, your portfolio rarely does, and backfill only needs to nibble away at history over time.
• Efficient caching — ingestion only reprocesses when files actually change (based on their modification time).

Storage & visualization

• InfluxDB 3 Core is the primary datastore. All live and historical points land in a single measurement, portfolio_metrics. See the InfluxDB data model for the full tag/field schema.
• Grafana connects to InfluxDB through its SQL query mode and ships with a provisioned dashboard in the Docker Compose stack.
• The legacy Prometheus endpoint (:8081/metrics) still exposes the sb_* gauges for the current snapshot of each share. See Legacy Prometheus endpoint.

Configuration modes

SuiviBourse supports two mutually exclusive configuration modes. See the Configuration section for details.

Mode	Source	Use case
Manual	config.yaml	Simple, static portfolio
Events	events/*.csv, *.xlsx	Transaction history, automatic aggregation, historical backfill