marco.chan
7923ba1b8d
- Create ingest-api with Koa framework for handling device telemetry. - Implement telemetry ingestion with MongoDB, Redis, and PostgreSQL. - Add API endpoints for telemetry and health checks. - Set up testing with Vitest for unit and end-to-end tests. - Configure Prettier and ESLint for code formatting and linting. - Create Docker Compose setup for PostgreSQL, MongoDB, and Redis services. - Establish contracts package for shared Zod schemas and TypeScript types. - Implement React frontend with device status and greeting functionalities. - Add VS Code setup documentation for onboarding developers.
94 lines
2.5 KiB
Markdown
94 lines
2.5 KiB
Markdown
# Architecture
|
|
|
|
Copy this layout for later device + dashboard projects.
|
|
|
|
## Lanes
|
|
|
|
- **Device lane:** `apps/ingest-api` (Koa) — firmware / gateways POST JSON.
|
|
- **User lane:** `apps/crud-api` (Express) — React, humans, billing later.
|
|
- **Shared types:** `packages/contracts` (Zod only). No Koa, Express, Mongo, or trest.
|
|
|
|
Frontend people can run `yarn workspaces focus web` and do not need the private trest registry.
|
|
|
|
## Data flow
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph devices [Device lane]
|
|
D[Device / demo ping]
|
|
I[Koa ingest-api :3001]
|
|
D -->|POST /telemetry JSON| I
|
|
end
|
|
|
|
subgraph stores [Stores]
|
|
M[(MongoDB raw telemetry)]
|
|
R[(Redis hot key TTL)]
|
|
P[(PostgreSQL device_current_state)]
|
|
A[(PostgreSQL analytics — not in this repo)]
|
|
end
|
|
|
|
subgraph humans [User lane]
|
|
W[React web :5173]
|
|
C[Express crud-api :3000]
|
|
W -->|axios GET /greeting GET /devices/:id/status| C
|
|
end
|
|
|
|
I -->|insert event| M
|
|
I -->|SET device:id EX 15s| R
|
|
I -->|UPSERT current row| P
|
|
M -.->|ETL cron later| A
|
|
C -->|SELECT snapshot| P
|
|
C -->|GET device:id| R
|
|
```
|
|
|
|
Near-real-time product path (Postgres + Express is enough at this scale):
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant D as Device
|
|
participant I as ingest-api Koa
|
|
participant PG as Postgres
|
|
participant C as crud-api Express
|
|
participant W as React
|
|
D->>I: POST /telemetry
|
|
I->>PG: UPSERT device_current_state
|
|
W->>C: GET /devices/dev-1/status poll 1s
|
|
C->>PG: SELECT by device_id
|
|
C->>W: lastSeen payload source postgres
|
|
```
|
|
|
|
Redis is compared on the Devices page. It is not required for the UI to look live.
|
|
|
|
## Demo dual-write (do not copy blindly)
|
|
|
|
Ingest writes Mongo first, then Redis and Postgres best-effort. A 200 can still mean Redis or Postgres missed. Production should derive those from a queue. This handler is a teaching shortcut.
|
|
|
|
Redis TTL expiry means “no recent ping.” Redis restart looks like everyone is offline. Postgres still has `device_current_state`.
|
|
|
|
Ingest may only upsert that snapshot table. CRUD owns greeting / future users and orgs.
|
|
|
|
Hourly Mongo → Postgres analytics is the dotted ETL box. Not implemented.
|
|
|
|
## Ports
|
|
|
|
| Service | Port |
|
|
| ---------- | ----- |
|
|
| crud-api | 3000 |
|
|
| ingest-api | 3001 |
|
|
| web | 5173 |
|
|
| postgres | 5432 |
|
|
| mongo | 27017 |
|
|
| redis | 6379 |
|
|
|
|
## Run
|
|
|
|
```bash
|
|
docker compose up -d
|
|
yarn install
|
|
yarn dev:crud
|
|
yarn dev:ingest
|
|
yarn dev:web
|
|
```
|
|
|
|
Open `http://127.0.0.1:5173/devices`, click **Ping dev-1**, watch Postgres and Redis columns. Wait 15s without pinging: Redis goes offline, Postgres keeps the last row.
|