0 Fork 0

Files

T

marco.chan 7923ba1b8d feat: initialize ingest-api with Koa and telemetry handling

- Create ingest-api with Koa framework for handling device telemetry.
- Implement telemetry ingestion with MongoDB, Redis, and PostgreSQL.
- Add API endpoints for telemetry and health checks.
- Set up testing with Vitest for unit and end-to-end tests.
- Configure Prettier and ESLint for code formatting and linting.
- Create Docker Compose setup for PostgreSQL, MongoDB, and Redis services.
- Establish contracts package for shared Zod schemas and TypeScript types.
- Implement React frontend with device status and greeting functionalities.
- Add VS Code setup documentation for onboarding developers.

2026-09-03 10:13:02 +08:00

2.5 KiB

Raw Blame History

Architecture

Copy this layout for later device + dashboard projects.

Lanes

Device lane: apps/ingest-api (Koa) — firmware / gateways POST JSON.
User lane: apps/crud-api (Express) — React, humans, billing later.
Shared types: packages/contracts (Zod only). No Koa, Express, Mongo, or trest.

Frontend people can run yarn workspaces focus web and do not need the private trest registry.

Data flow

flowchart LR
  subgraph devices [Device lane]
    D[Device / demo ping]
    I[Koa ingest-api :3001]
    D -->|POST /telemetry JSON| I
  end

  subgraph stores [Stores]
    M[(MongoDB raw telemetry)]
    R[(Redis hot key TTL)]
    P[(PostgreSQL device_current_state)]
    A[(PostgreSQL analytics — not in this repo)]
  end

  subgraph humans [User lane]
    W[React web :5173]
    C[Express crud-api :3000]
    W -->|axios GET /greeting GET /devices/:id/status| C
  end

  I -->|insert event| M
  I -->|SET device:id EX 15s| R
  I -->|UPSERT current row| P
  M -.->|ETL cron later| A
  C -->|SELECT snapshot| P
  C -->|GET device:id| R

Near-real-time product path (Postgres + Express is enough at this scale):

sequenceDiagram
  participant D as Device
  participant I as ingest-api Koa
  participant PG as Postgres
  participant C as crud-api Express
  participant W as React
  D->>I: POST /telemetry
  I->>PG: UPSERT device_current_state
  W->>C: GET /devices/dev-1/status poll 1s
  C->>PG: SELECT by device_id
  C->>W: lastSeen payload source postgres

Redis is compared on the Devices page. It is not required for the UI to look live.

Demo dual-write (do not copy blindly)

Ingest writes Mongo first, then Redis and Postgres best-effort. A 200 can still mean Redis or Postgres missed. Production should derive those from a queue. This handler is a teaching shortcut.

Redis TTL expiry means “no recent ping.” Redis restart looks like everyone is offline. Postgres still has device_current_state.

Ingest may only upsert that snapshot table. CRUD owns greeting / future users and orgs.

Hourly Mongo → Postgres analytics is the dotted ETL box. Not implemented.

Ports

Service	Port
crud-api	3000
ingest-api	3001
web	5173
postgres	5432
mongo	27017
redis	6379

Run

docker compose up -d
yarn install
yarn dev:crud
yarn dev:ingest
yarn dev:web

Open http://127.0.0.1:5173/devices, click Ping dev-1, watch Postgres and Redis columns. Wait 15s without pinging: Redis goes offline, Postgres keeps the last row.

2.5 KiB Raw Blame History