Back office: Postgres source-of-truth + read dashboard

Stand up the DB-first CRM backbone (architecture pivot: Postgres is the
source of truth, Google Sheets becomes a one-way downstream mirror).

- backoffice/ stack: smb-db (Postgres 16) + smb-crm (Flask/waitress service).
- Schema mirrors the six Sheet tabs (clients, leads, projects, activity_log,
  bookings, invoices) with typed columns + updated_at triggers.
- Service-account Sheets client (PyJWT) for the one-time import + future mirror.
- import_from_sheets.py: idempotent seed of Postgres from the live Sheets.
- Read dashboard (Leads & Clients tables) at onboard.mivanchenko.de/crm,
  behind the existing Caddy basic-auth; JSON API reads straight from Postgres.

Deployed + verified: import seeded DB, dashboard/API live, no-auth blocked,
onboarding form unaffected. Add/edit/delete + DB->Sheets sync + n8n ingest
swap are the next steps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-25 09:12:39 +02:00
parent a04ad82e73
commit 18e346d67b
11 changed files with 661 additions and 0 deletions
+55
View File
@@ -0,0 +1,55 @@
"""One-time (idempotent) seed of Postgres from the existing Google Sheets.
This is the ONLY place the Sheet is treated as authoritative — to bootstrap the
DB. After this, Postgres is the source of truth and Sheets is a downstream
mirror. Safe to re-run: keyed tables upsert by primary key; activity_log is
replaced wholesale.
Run inside the container: docker compose exec smb-crm python import_from_sheets.py
"""
import os
import sys
import db
from sheets import Sheets
def upsert(cur, entity, records):
spec = db.TABLES[entity]
cols = spec["cols"]
pk = spec["pk"]
n = 0
for rec in records:
row = db.coerce_row(entity, rec)
if entity != "activity_log" and not row.get(pk):
continue # skip rows without a primary key
placeholders = ", ".join(["%s"] * len(cols))
collist = ", ".join(cols)
if entity == "activity_log":
cur.execute(f"INSERT INTO {entity} ({collist}) VALUES ({placeholders})",
[row[c] for c in cols])
else:
updates = ", ".join(f"{c} = EXCLUDED.{c}" for c in cols if c != pk)
cur.execute(
f"INSERT INTO {entity} ({collist}) VALUES ({placeholders}) "
f"ON CONFLICT ({pk}) DO UPDATE SET {updates}",
[row[c] for c in cols])
n += 1
return n
def main():
sh = Sheets(os.environ["GOOGLE_SA_JSON"], os.environ["SHEET_ID"])
with db.connect() as conn:
with conn.cursor() as cur:
cur.execute("TRUNCATE activity_log RESTART IDENTITY")
for entity, spec in db.TABLES.items():
records = sh.read_records(spec["tab"])
n = upsert(cur, entity, records)
print(f" {entity:<13} <- {spec['tab']:<13} {n} row(s)")
conn.commit()
print("import complete.")
if __name__ == "__main__":
sys.exit(main())