Goa-gel-fullstack/backend/DATABASE_SETUP.md

# Goa GEL Database Setup - Complete Guide

## Overview

This document provides a comprehensive overview of the complete database setup for the Goa GEL Blockchain Document Verification Platform, including all 12 entities, complete migrations, seeders, and configuration.

## Created Files Summary

### 1. Entity Files (12 entities)

All entities are located in `/src/database/entities/`:

| Entity | File | Purpose |
|--------|------|---------|
| Applicant | `applicant.entity.ts` | Represents license applicants with wallet integration |
| Department | `department.entity.ts` | Government departments handling approvals |
| Workflow | `workflow.entity.ts` | Multi-stage approval workflow definitions |
| LicenseRequest | `license-request.entity.ts` | Main license application entity (8 statuses) |
| Document | `document.entity.ts` | Uploaded documents with versioning support |
| DocumentVersion | `document-version.entity.ts` | Version history for documents (SHA-256 hashing) |
| Approval | `approval.entity.ts` | Department-level approvals (5 statuses) |
| WorkflowState | `workflow-state.entity.ts` | Execution state tracking with full audit |
| Webhook | `webhook.entity.ts` | Department webhook configurations |
| WebhookLog | `webhook-log.entity.ts` | Webhook delivery audit trail (retry tracking) |
| AuditLog | `audit-log.entity.ts` | Comprehensive change audit with actor tracking |
| BlockchainTransaction | `blockchain-transaction.entity.ts` | NFT minting and on-chain operations (5 tx types) |

**Index File:** `entities/index.ts` - Exports all entities and enums

### 2. Core Configuration

- **data-source.ts** - TypeORM DataSource with PostgreSQL configuration
  - Environment variable driven
  - Connection pooling configured
  - Logging support for development
  - All 12 entities registered

### 3. Migrations

- **1704067200000-InitialSchema.ts** - Complete initial schema
  - 12 tables created with proper constraints
  - 7 custom PostgreSQL enums
  - 40+ indexes for performance optimization
  - Foreign key relationships with cascade delete
  - Complete down migration for rollback

### 4. Database Seeders

- **seeders/seed.ts** - Sample data generator
  - Creates 4 sample departments (Fire, Tourism, Municipal, Health)
  - Defines RESORT_LICENSE workflow with 5 stages
  - Creates 2 sample applicants
  - Creates 1 license request in DRAFT status
  - Generates workflow state with execution log

### 5. Documentation

- **src/database/README.md** - Comprehensive database documentation
- **DATABASE_SETUP.md** - This file

## Database Schema Details

### Table: applicants

```typescript
id: UUID (PK)
digilockerId: varchar(255) UNIQUE
name: varchar(255)
email: varchar(255) UNIQUE
phone: varchar(20)
walletAddress: varchar(255) UNIQUE
isActive: boolean (default: true)
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - digilockerId
  - walletAddress
  - email
```

### Table: departments

```typescript
id: UUID (PK)
code: varchar(50) UNIQUE
name: varchar(255)
walletAddress: varchar(255) UNIQUE
apiKeyHash: varchar(255)
apiSecretHash: varchar(255)
webhookUrl: text NULLABLE
webhookSecretHash: varchar(255) NULLABLE
isActive: boolean (default: true)
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - code
  - walletAddress
```

### Table: workflows

```typescript
id: UUID (PK)
workflowType: varchar(100) UNIQUE
name: varchar(255)
description: text NULLABLE
version: integer (default: 1)
definition: jsonb
isActive: boolean (default: true)
createdBy: UUID NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - workflowType
```

### Table: license_requests

```typescript
id: UUID (PK)
requestNumber: varchar(50) UNIQUE (auto-generated: RL-YYYY-XXXXXX)
tokenId: bigint NULLABLE
applicantId: UUID (FK: applicants)
requestType: varchar(50)
workflowId: UUID (FK: workflows)
status: ENUM (DRAFT, SUBMITTED, IN_REVIEW, PENDING_RESUBMISSION, APPROVED, REJECTED, REVOKED, CANCELLED)
metadata: jsonb (default: {})
currentStageId: varchar(100) NULLABLE
blockchainTxHash: varchar(255) NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE
submittedAt: TIMESTAMP WITH TIME ZONE NULLABLE
approvedAt: TIMESTAMP WITH TIME ZONE NULLABLE

Indexes:
  - requestNumber
  - applicantId
  - workflowId
  - status
  - createdAt
  - (applicantId, status)
```

### Table: documents

```typescript
id: UUID (PK)
requestId: UUID (FK: license_requests)
docType: varchar(100)
originalFilename: varchar(255)
currentVersion: integer (default: 1)
currentHash: varchar(64) [SHA-256]
minioBucket: varchar(255)
isActive: boolean (default: true)
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - requestId
  - (requestId, docType)
  - currentHash
```

### Table: document_versions

```typescript
id: UUID (PK)
documentId: UUID (FK: documents)
version: integer
hash: varchar(64) [SHA-256]
minioPath: text
fileSize: bigint
mimeType: varchar(100)
uploadedBy: UUID
blockchainTxHash: varchar(255) NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - documentId
  - hash

Unique Constraint:
  - (documentId, version)
```

### Table: approvals

```typescript
id: UUID (PK)
requestId: UUID (FK: license_requests)
departmentId: UUID (FK: departments)
status: ENUM (PENDING, APPROVED, REJECTED, CHANGES_REQUESTED, REVIEW_REQUIRED)
remarks: text NULLABLE
remarksHash: varchar(64) NULLABLE [SHA-256]
reviewedDocuments: jsonb (array of UUIDs, default: [])
blockchainTxHash: varchar(255) NULLABLE
isActive: boolean (default: true)
invalidatedAt: TIMESTAMP WITH TIME ZONE NULLABLE
invalidationReason: varchar(255) NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - requestId
  - departmentId
  - status
  - (requestId, departmentId)
  - (requestId, status)
```

### Table: workflow_states

```typescript
id: UUID (PK)
requestId: UUID (FK: license_requests) UNIQUE
currentStageId: varchar(100)
completedStages: jsonb (array of stage IDs, default: [])
pendingApprovals: jsonb (array of {departmentCode, status, createdAt}, default: [])
executionLog: jsonb (array of {timestamp, stageId, action, details}, default: [])
stageStartedAt: TIMESTAMP WITH TIME ZONE NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - requestId
```

### Table: webhooks

```typescript
id: UUID (PK)
departmentId: UUID (FK: departments)
url: text
events: jsonb (array of event types, default: [])
secretHash: varchar(255)
isActive: boolean (default: true)
createdAt: TIMESTAMP WITH TIME ZONE
updatedAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - departmentId
  - (departmentId, isActive)
```

### Table: webhook_logs

```typescript
id: UUID (PK)
webhookId: UUID (FK: webhooks)
eventType: varchar(100)
payload: jsonb
responseStatus: integer NULLABLE
responseBody: text NULLABLE
responseTime: integer NULLABLE [milliseconds]
retryCount: integer (default: 0)
status: ENUM (PENDING, SUCCESS, FAILED)
createdAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - webhookId
  - eventType
  - status
  - createdAt
  - (webhookId, status)
```

### Table: audit_logs

```typescript
id: UUID (PK)
entityType: ENUM (REQUEST, APPROVAL, DOCUMENT, DEPARTMENT, WORKFLOW)
entityId: UUID
action: varchar(100)
actorType: ENUM (APPLICANT, DEPARTMENT, SYSTEM, ADMIN)
actorId: UUID NULLABLE
oldValue: jsonb NULLABLE
newValue: jsonb NULLABLE
ipAddress: varchar(45) NULLABLE
userAgent: text NULLABLE
correlationId: varchar(255) NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE

Indexes:
  - entityType
  - entityId
  - action
  - actorType
  - createdAt
  - (entityType, entityId)
  - (actorId, createdAt)
```

### Table: blockchain_transactions

```typescript
id: UUID (PK)
txHash: varchar(255) UNIQUE
txType: ENUM (MINT_NFT, APPROVAL, DOC_UPDATE, REJECT, REVOKE)
relatedEntityType: varchar(100)
relatedEntityId: UUID
fromAddress: varchar(255)
toAddress: varchar(255) NULLABLE
status: ENUM (PENDING, CONFIRMED, FAILED)
blockNumber: bigint NULLABLE
gasUsed: bigint NULLABLE
errorMessage: text NULLABLE
createdAt: TIMESTAMP WITH TIME ZONE
confirmedAt: TIMESTAMP WITH TIME ZONE NULLABLE

Indexes:
  - txHash
  - status
  - txType
  - relatedEntityId
  - createdAt
  - (status, txType)
```

## Enums Defined

### LicenseRequestStatus (license_requests table)
- DRAFT
- SUBMITTED
- IN_REVIEW
- PENDING_RESUBMISSION
- APPROVED
- REJECTED
- REVOKED
- CANCELLED

### ApprovalStatus (approvals table)
- PENDING
- APPROVED
- REJECTED
- CHANGES_REQUESTED
- REVIEW_REQUIRED

### WebhookLogStatus (webhook_logs table)
- PENDING
- SUCCESS
- FAILED

### AuditEntityType (audit_logs table)
- REQUEST
- APPROVAL
- DOCUMENT
- DEPARTMENT
- WORKFLOW

### AuditActorType (audit_logs table)
- APPLICANT
- DEPARTMENT
- SYSTEM
- ADMIN

### BlockchainTransactionType (blockchain_transactions table)
- MINT_NFT
- APPROVAL
- DOC_UPDATE
- REJECT
- REVOKE

### BlockchainTransactionStatus (blockchain_transactions table)
- PENDING
- CONFIRMED
- FAILED

## Key Features

### 1. Data Integrity
- All foreign keys with CASCADE DELETE
- UNIQUE constraints on critical fields
- NOT NULL constraints where required
- CHECK constraints via TypeORM validation

### 2. Performance Optimization
- 40+ indexes covering all query patterns
- Composite indexes for common joins
- JSONB columns for flexible metadata
- Partitioning ready for large audit tables

### 3. Audit & Compliance
- Complete audit_logs tracking all changes
- Actor identification (who made the change)
- Old/new values for change comparison
- Correlation IDs for distributed tracing
- IP address and user agent capture

### 4. Blockchain Integration
- blockchain_transactions table for on-chain tracking
- NFT token ID field in license_requests
- Transaction hash storage for verification
- Block confirmation tracking

### 5. Workflow Management
- workflow_states for execution tracking
- Complete execution log with timestamps
- Pending approvals tracking by department
- Completed stages audit trail

### 6. Document Versioning
- Multiple versions per document
- SHA-256 hashing for integrity
- File size and mime type tracking
- Upload attribution and timestamps

### 7. Webhook System
- Flexible event subscription model
- Retry mechanism with count tracking
- Response time monitoring
- Status tracking (PENDING, SUCCESS, FAILED)

## Setup Instructions

### Prerequisites
- PostgreSQL 12+ with UUID extension
- Node.js 16+ with TypeORM
- npm or yarn package manager

### Steps

1. **Install Dependencies**
   ```bash
   npm install typeorm pg uuid crypto dotenv
   ```

2. **Create PostgreSQL Database**
   ```bash
   createdb goa_gel_db
   ```

3. **Configure Environment**
   Create `.env` file:
   ```env
   DATABASE_HOST=localhost
   DATABASE_PORT=5432
   DATABASE_USER=postgres
   DATABASE_PASSWORD=your_password
   DATABASE_NAME=goa_gel_db
   DATABASE_LOGGING=true
   DATABASE_SSL=false
   NODE_ENV=development
   ```

4. **Run Migrations**
   ```bash
   npx typeorm migration:run -d src/database/data-source.ts
   ```

5. **Seed Sample Data**
   ```bash
   npx ts-node src/database/seeders/seed.ts
   ```

6. **Verify Setup**
   ```bash
   psql goa_gel_db -c "\dt"
   ```

## Entity Relationships

```
┌─────────────┐
│ Applicant   │
└──────┬──────┘
       │ 1:N
       │
       ├─────────────────────────────┬──────────────────┐
       │                             │                  │
       ▼                             ▼                  ▼
┌──────────────────┐      ┌──────────────┐     ┌────────────────┐
│ LicenseRequest   │      │  Workflow    │     │ WorkflowState  │
└────┬─────────────┘      └──────────────┘     └────────────────┘
     │                                                    △
     │ 1:N                                               │
     ├─────────┬──────────┐                              │
     │         │          │                              │
     ▼         ▼          ▼                          1:1 relation
┌────────────┐ ┌───────────┐ ┌──────────────┐
│ Document   │ │ Approval  │ │ Approval     │
│ 1:N        │ │ 1:N       │ │ Status       │
│ DocumentV  │ │ Department│ │ Tracking     │
└────────────┘ └─────┬─────┘ └──────────────┘
               │
               │ N:1
               │
        ┌──────▼──────┐
        │ Department  │
        │ 1:N         │
        │ Webhook     │
        │ 1:N         │
        │ WebhookLog  │
        └─────────────┘

AuditLog ─── Tracks all changes to above entities

BlockchainTransaction ─── Records NFT minting and approvals
```

## Common Operations

### Create a New Migration
```bash
npx typeorm migration:generate -d src/database/data-source.ts -n AddNewField
```

### Generate Migration from Entity Changes
```bash
npx typeorm migration:generate -d src/database/data-source.ts -n AutoGenerated
```

### Revert Last Migration
```bash
npx typeorm migration:revert -d src/database/data-source.ts
```

### View Migration Status
```bash
npx typeorm migration:show -d src/database/data-source.ts
```

## Security Considerations

1. **Hash Storage** - API keys, secrets, and webhook secrets are hashed with SHA-256
2. **Wallet Addresses** - Normalized to lowercase to prevent duplication
3. **Cascade Delete** - Foreign keys cascade to prevent orphaned records
4. **Audit Trail** - All critical operations logged with actor identification
5. **Correlation IDs** - Support distributed tracing for audit
6. **JSONB Validation** - Additional validation at application layer

## Performance Tips

1. **Index Usage** - All frequently queried columns are indexed
2. **Composite Indexes** - Multi-column queries optimized
3. **JSONB Queries** - Use PostgreSQL native JSONB operations
4. **Batch Operations** - Use chunking for large inserts
5. **Connection Pooling** - Configured at 20 connections (production)

## File Locations

```
/sessions/cool-elegant-faraday/mnt/Goa-GEL/backend/src/database/
├── entities/
│   ├── applicant.entity.ts
│   ├── approval.entity.ts
│   ├── audit-log.entity.ts
│   ├── blockchain-transaction.entity.ts
│   ├── department.entity.ts
│   ├── document-version.entity.ts
│   ├── document.entity.ts
│   ├── index.ts
│   ├── license-request.entity.ts
│   ├── webhook-log.entity.ts
│   ├── webhook.entity.ts
│   ├── workflow-state.entity.ts
│   └── workflow.entity.ts
├── migrations/
│   └── 1704067200000-InitialSchema.ts
├── seeders/
│   └── seed.ts
├── data-source.ts
├── index.ts
└── README.md
```

## Next Steps

1. Install PostgreSQL and create database
2. Configure environment variables in `.env`
3. Run migrations: `npx typeorm migration:run -d src/database/data-source.ts`
4. Seed sample data: `npx ts-node src/database/seeders/seed.ts`
5. Start backend application
6. Verify database tables: `psql goa_gel_db -c "\dt"`

## Support

For detailed information, see:
- `/src/database/README.md` - Database documentation
- `/src/database/entities/` - Entity definitions with comments
- `/src/database/migrations/` - SQL migration details