Architecture Reference

Technical reference for contributors and those interested in Hub's internal design.

Audience

This page is for contributors and advanced users who want to understand Hub's internals. If you're just using Hub, see the Quick Start Guide instead.

Technology Stack

Language: Go 1.22+
Web Framework: Huma v2 (OpenAPI-first) + Chi router
ORM: Ent (type-safe, code generation)
Database: PostgreSQL 18
Documentation: OpenAPI 3.1 + Scalar
Testing: Go's built-in testing + testcontainers-go
License: Apache 2.0

Project Structure

apps/hub/
├── cmd/
│   └── hub/
│       └── main.go              # Entry point, CLI setup
├── internal/
│   ├── api/
│   │   ├── server.go            # HTTP server, Huma setup
│   │   ├── experiences.go       # CRUD handlers
│   │   └── types.go             # API request/response types
│   ├── config/
│   │   └── config.go            # Configuration management
│   ├── ent/
│   │   ├── schema/
│   │   │   └── experiencedata.go  # Ent schema definition
│   │   └── [generated files]       # Auto-generated by Ent
│   ├── middleware/
│   │   ├── auth.go              # API key authentication
│   │   ├── logging.go           # Structured logging
│   │   └── middleware.go        # Shared middleware
│   ├── models/
│   │   └── experience.go        # Domain models (business logic)
│   └── webhook/
│       ├── dispatcher.go        # Webhook delivery
│       └── dispatcher_test.go   # Webhook tests
├── docker-compose.yml           # Local PostgreSQL
├── Dockerfile                   # Production image
├── Makefile                     # Dev commands
├── go.mod                       # Go dependencies
└── README.md                    # Developer docs

Architecture Layers

1. Entry Point (`cmd/hub/main.go`)

Parses CLI arguments and environment variables using Huma CLI
Initializes logger based on SERVICE_LOG_LEVEL
Connects to PostgreSQL via Ent
Runs database migrations automatically
Creates and starts HTTP server

2. API Layer (`internal/api/`)

Responsibilities:

HTTP request handling
Request validation
Response formatting
OpenAPI documentation generation

Key Components:

server.go - Huma API setup, Chi router, middleware registration
experiences.go - CRUD operation handlers
types.go - API-specific DTOs (Data Transfer Objects)

3. Middleware (`internal/middleware/`)

Chain:

Chi middleware (RequestID, RealIP, Recoverer)
Custom logging middleware (structured logs)
Optional API key auth middleware

Execution order:

Request → Chi middleware → Huma API → Custom middleware → Handler → Response

4. Domain Models (`internal/models/`)

Purpose: Separate API concerns from business logic.

Flow:

API Request → API DTO → Domain Model → Ent Entity → Database
                ↓                           ↑
            Validation              Database Query

Why?

Clean separation of concerns
Easy to add business logic (e.g., calculated fields)
API changes don't directly affect database schema

5. ORM Layer (`internal/ent/`)

Ent's role:

Type-safe database queries
Auto-generate CRUD methods
Handle migrations automatically
Provide graph traversal (for future relations)

Code generation workflow:

Define schema in internal/ent/schema/experiencedata.go
Run make ent-gen to generate code
Generated files in internal/ent/ (committed to Git)

6. Webhook System (`internal/webhook/`)

Design:

Async dispatch (doesn't block API responses)
Retry logic with exponential backoff
Concurrent delivery to multiple URLs

Delivery guarantees:

At-least-once delivery (with retries)
5-second timeout per attempt
3 total attempts (1 initial + 2 retries)

Data Flow

Create Experience

1. POST /v1/experiences
   ↓
2. Huma validates request against OpenAPI spec
   ↓
3. Handler receives CreateExperienceInput
   ↓
4. Convert API DTO to Ent builder
   ↓
5. Save to PostgreSQL via Ent
   ↓
6. Convert Ent entity → Domain model → API response
   ↓
7. Dispatch webhook asynchronously
   ↓
8. Return 201 Created with experience data

List Experiences with Filters

1. GET /v1/experiences?source_type=survey&limit=50
   ↓
2. Huma parses and validates query parameters
   ↓
3. Handler builds Ent query with filters
   ↓
4. Execute query with pagination
   ↓
5. Get total count (separate query)
   ↓
6. Convert results to API format
   ↓
7. Return paginated response

Database Schema

ExperienceData Table

CREATE TABLE experience_data (
  -- Primary key
  id UUID PRIMARY KEY DEFAULT gen_uuidv7(),
  
  -- Timestamps
  collected_at TIMESTAMP NOT NULL DEFAULT NOW(),
  created_at TIMESTAMP NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMP NOT NULL DEFAULT NOW(),
  
  -- Source information
  source_type VARCHAR NOT NULL,
  source_id VARCHAR,
  source_name VARCHAR,
  
  -- Field information
  field_id VARCHAR NOT NULL,
  field_label VARCHAR,
  field_type VARCHAR NOT NULL,
  
  -- Response values
  value_text TEXT,
  value_number DOUBLE PRECISION,
  value_boolean BOOLEAN,
  value_date TIMESTAMP,
  value_json JSONB,
  
  -- Metadata
  metadata JSONB,
  language VARCHAR(10),
  user_identifier VARCHAR
);

-- Indexes for analytics
CREATE INDEX idx_source_type ON experience_data(source_type);
CREATE INDEX idx_source_id ON experience_data(source_id);
CREATE INDEX idx_collected_at ON experience_data(collected_at);
CREATE INDEX idx_field_type ON experience_data(field_type);
CREATE INDEX idx_value_number ON experience_data(value_number);
CREATE INDEX idx_user_identifier ON experience_data(user_identifier);

-- Composite indexes
CREATE INDEX idx_source_collected ON experience_data(source_type, source_id, collected_at);
CREATE INDEX idx_field_collected ON experience_data(field_type, collected_at);

Configuration Management

Huma CLI Integration

Hub uses Huma's built-in CLI with the SERVICE_ prefix convention:

type Options struct {
    Port        int    `help:"Port to listen on" default:"8080"`
    DatabaseURL string `help:"PostgreSQL connection URL" required:"true"`
    APIKey      string `help:"Optional API key" default:""`
    // ...
}

Environment variable mapping:

SERVICE_PORT → Options.Port
SERVICE_DATABASE_URL → Options.DatabaseURL
SERVICE_API_KEY → Options.APIKey

Testing Strategy

Unit Tests

Test individual functions:

// internal/webhook/dispatcher_test.go
func TestDispatcher_Dispatch(t *testing.T) {
    // Test webhook delivery logic
}

Integration Tests

Test API endpoints with real database:

// internal/api/experiences_test.go
func TestCreateExperience(t *testing.T) {
    // Uses testcontainers-go for real PostgreSQL
    // Tests full request → response flow
}

Why testcontainers?

✅ Test against real PostgreSQL (production parity)
✅ Automatic cleanup after tests
✅ CI/CD friendly (GitHub Actions)
✅ No mocking database behavior

Development Workflow

Making Changes

Edit code in internal/
Hot reload via Air (automatic restart)
Test locally against PostgreSQL in Docker
Run tests: make test
Check types: go build ./cmd/hub

Adding a New Field to ExperienceData

Edit internal/ent/schema/experiencedata.go
Run make ent-gen to regenerate code
Restart service (migrations run automatically)
Update API types in internal/api/types.go
Update handlers in internal/api/experiences.go
Update tests

Adding a New Endpoint

Define operation in internal/api/experiences.go:

huma.Register(api, huma.Operation{
    OperationID: "my-operation",
    Method:      "GET",
    Path:        "/v1/my-endpoint",
    // ...
}, handlerFunc)

Create input/output types in internal/api/types.go
Implement handler logic
Write tests in internal/api/*_test.go
OpenAPI docs auto-generate

Performance Considerations

Database Indexes

Pre-created indexes optimize common queries:

Time-range queries (collected_at)
Filtering by source (source_type, source_id)
Numeric aggregations (value_number)

Connection Pooling

PostgreSQL driver handles connection pooling automatically. Configure via connection string:

?pool_max_conns=20&pool_min_conns=5

Webhook Async Delivery

Webhooks don't block API responses. They're dispatched in goroutines with timeout and retry logic.

Security

API Key Authentication

Simple shared-secret authentication via X-API-Key header. For production, consider:

API Gateway (AWS API Gateway, Kong, Traefik)
OAuth 2.0 / JWT
mTLS for service-to-service

Database Security

Use parameterized queries (Ent does this automatically)
Never log sensitive data
Use sslmode=require in production

Observability

Structured Logging

All logs use slog (Go 1.21+):

{
  "time": "2025-10-15T12:34:56Z",
  "level": "INFO",
  "msg": "experience created",
  "id": "01932c8a-8b9e-7000-8000-000000000001",
  "source_type": "survey"
}

Health Checks

GET /health - Always returns {"status":"ok"}
Database connectivity checked on startup only

Contributing

See CONTRIBUTING.md for:

Code style guidelines
Pull request process
Testing requirements
Community guidelines

Next Steps for Contributors

Set up locally: Quick Start Guide
Explore codebase: apps/hub/internal/
Run tests: make test
GitHub Discussions: Ask questions

Technology Stack​

Project Structure​

Architecture Layers​

1. Entry Point (cmd/hub/main.go)​

2. API Layer (internal/api/)​

3. Middleware (internal/middleware/)​

4. Domain Models (internal/models/)​

5. ORM Layer (internal/ent/)​

6. Webhook System (internal/webhook/)​

Data Flow​

Create Experience​

List Experiences with Filters​

Database Schema​

ExperienceData Table​

Configuration Management​

Huma CLI Integration​

Testing Strategy​

Unit Tests​

Integration Tests​

Development Workflow​

Making Changes​

Adding a New Field to ExperienceData​

Adding a New Endpoint​

Performance Considerations​

Database Indexes​

Connection Pooling​

Webhook Async Delivery​

Security​

API Key Authentication​

Database Security​

Observability​

Structured Logging​

Health Checks​

Contributing​

Next Steps for Contributors​