feat: Goa GEL Blockchain e-Licensing Platform - Full Stack Implementation
Complete implementation of the Goa Government e-Licensing platform with: Backend: - NestJS API with JWT authentication - PostgreSQL database with Knex ORM - Redis caching and session management - MinIO document storage - Hyperledger Besu blockchain integration - Multi-department workflow system - Comprehensive API tests (266/282 passing) Frontend: - Angular 21 with standalone components - Angular Material + TailwindCSS UI - Visual workflow builder - Document upload with progress tracking - Blockchain explorer integration - Role-based dashboards (Admin, Department, Citizen) - E2E tests with Playwright (37 tests) Infrastructure: - Docker Compose orchestration - Blockscout blockchain explorer - Development and production configurations
This commit is contained in:
Mahi
583
backend/DATABASE_SETUP.md
Normal file
583
backend/DATABASE_SETUP.md
Normal file
@@ -0,0 +1,583 @@
|
||||
# Goa GEL Database Setup - Complete Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a comprehensive overview of the complete database setup for the Goa GEL Blockchain Document Verification Platform, including all 12 entities, complete migrations, seeders, and configuration.
|
||||
|
||||
## Created Files Summary
|
||||
|
||||
### 1. Entity Files (12 entities)
|
||||
|
||||
All entities are located in `/src/database/entities/`:
|
||||
|
||||
| Entity | File | Purpose |
|
||||
|--------|------|---------|
|
||||
| Applicant | `applicant.entity.ts` | Represents license applicants with wallet integration |
|
||||
| Department | `department.entity.ts` | Government departments handling approvals |
|
||||
| Workflow | `workflow.entity.ts` | Multi-stage approval workflow definitions |
|
||||
| LicenseRequest | `license-request.entity.ts` | Main license application entity (8 statuses) |
|
||||
| Document | `document.entity.ts` | Uploaded documents with versioning support |
|
||||
| DocumentVersion | `document-version.entity.ts` | Version history for documents (SHA-256 hashing) |
|
||||
| Approval | `approval.entity.ts` | Department-level approvals (5 statuses) |
|
||||
| WorkflowState | `workflow-state.entity.ts` | Execution state tracking with full audit |
|
||||
| Webhook | `webhook.entity.ts` | Department webhook configurations |
|
||||
| WebhookLog | `webhook-log.entity.ts` | Webhook delivery audit trail (retry tracking) |
|
||||
| AuditLog | `audit-log.entity.ts` | Comprehensive change audit with actor tracking |
|
||||
| BlockchainTransaction | `blockchain-transaction.entity.ts` | NFT minting and on-chain operations (5 tx types) |
|
||||
|
||||
**Index File:** `entities/index.ts` - Exports all entities and enums
|
||||
|
||||
### 2. Core Configuration
|
||||
|
||||
- **data-source.ts** - TypeORM DataSource with PostgreSQL configuration
|
||||
- Environment variable driven
|
||||
- Connection pooling configured
|
||||
- Logging support for development
|
||||
- All 12 entities registered
|
||||
|
||||
### 3. Migrations
|
||||
|
||||
- **1704067200000-InitialSchema.ts** - Complete initial schema
|
||||
- 12 tables created with proper constraints
|
||||
- 7 custom PostgreSQL enums
|
||||
- 40+ indexes for performance optimization
|
||||
- Foreign key relationships with cascade delete
|
||||
- Complete down migration for rollback
|
||||
|
||||
### 4. Database Seeders
|
||||
|
||||
- **seeders/seed.ts** - Sample data generator
|
||||
- Creates 4 sample departments (Fire, Tourism, Municipal, Health)
|
||||
- Defines RESORT_LICENSE workflow with 5 stages
|
||||
- Creates 2 sample applicants
|
||||
- Creates 1 license request in DRAFT status
|
||||
- Generates workflow state with execution log
|
||||
|
||||
### 5. Documentation
|
||||
|
||||
- **src/database/README.md** - Comprehensive database documentation
|
||||
- **DATABASE_SETUP.md** - This file
|
||||
|
||||
## Database Schema Details
|
||||
|
||||
### Table: applicants
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
digilockerId: varchar(255) UNIQUE
|
||||
name: varchar(255)
|
||||
email: varchar(255) UNIQUE
|
||||
phone: varchar(20)
|
||||
walletAddress: varchar(255) UNIQUE
|
||||
isActive: boolean (default: true)
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- digilockerId
|
||||
- walletAddress
|
||||
- email
|
||||
```
|
||||
|
||||
### Table: departments
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
code: varchar(50) UNIQUE
|
||||
name: varchar(255)
|
||||
walletAddress: varchar(255) UNIQUE
|
||||
apiKeyHash: varchar(255)
|
||||
apiSecretHash: varchar(255)
|
||||
webhookUrl: text NULLABLE
|
||||
webhookSecretHash: varchar(255) NULLABLE
|
||||
isActive: boolean (default: true)
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- code
|
||||
- walletAddress
|
||||
```
|
||||
|
||||
### Table: workflows
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
workflowType: varchar(100) UNIQUE
|
||||
name: varchar(255)
|
||||
description: text NULLABLE
|
||||
version: integer (default: 1)
|
||||
definition: jsonb
|
||||
isActive: boolean (default: true)
|
||||
createdBy: UUID NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- workflowType
|
||||
```
|
||||
|
||||
### Table: license_requests
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
requestNumber: varchar(50) UNIQUE (auto-generated: RL-YYYY-XXXXXX)
|
||||
tokenId: bigint NULLABLE
|
||||
applicantId: UUID (FK: applicants)
|
||||
requestType: varchar(50)
|
||||
workflowId: UUID (FK: workflows)
|
||||
status: ENUM (DRAFT, SUBMITTED, IN_REVIEW, PENDING_RESUBMISSION, APPROVED, REJECTED, REVOKED, CANCELLED)
|
||||
metadata: jsonb (default: {})
|
||||
currentStageId: varchar(100) NULLABLE
|
||||
blockchainTxHash: varchar(255) NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
submittedAt: TIMESTAMP WITH TIME ZONE NULLABLE
|
||||
approvedAt: TIMESTAMP WITH TIME ZONE NULLABLE
|
||||
|
||||
Indexes:
|
||||
- requestNumber
|
||||
- applicantId
|
||||
- workflowId
|
||||
- status
|
||||
- createdAt
|
||||
- (applicantId, status)
|
||||
```
|
||||
|
||||
### Table: documents
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
requestId: UUID (FK: license_requests)
|
||||
docType: varchar(100)
|
||||
originalFilename: varchar(255)
|
||||
currentVersion: integer (default: 1)
|
||||
currentHash: varchar(64) [SHA-256]
|
||||
minioBucket: varchar(255)
|
||||
isActive: boolean (default: true)
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- requestId
|
||||
- (requestId, docType)
|
||||
- currentHash
|
||||
```
|
||||
|
||||
### Table: document_versions
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
documentId: UUID (FK: documents)
|
||||
version: integer
|
||||
hash: varchar(64) [SHA-256]
|
||||
minioPath: text
|
||||
fileSize: bigint
|
||||
mimeType: varchar(100)
|
||||
uploadedBy: UUID
|
||||
blockchainTxHash: varchar(255) NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- documentId
|
||||
- hash
|
||||
|
||||
Unique Constraint:
|
||||
- (documentId, version)
|
||||
```
|
||||
|
||||
### Table: approvals
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
requestId: UUID (FK: license_requests)
|
||||
departmentId: UUID (FK: departments)
|
||||
status: ENUM (PENDING, APPROVED, REJECTED, CHANGES_REQUESTED, REVIEW_REQUIRED)
|
||||
remarks: text NULLABLE
|
||||
remarksHash: varchar(64) NULLABLE [SHA-256]
|
||||
reviewedDocuments: jsonb (array of UUIDs, default: [])
|
||||
blockchainTxHash: varchar(255) NULLABLE
|
||||
isActive: boolean (default: true)
|
||||
invalidatedAt: TIMESTAMP WITH TIME ZONE NULLABLE
|
||||
invalidationReason: varchar(255) NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- requestId
|
||||
- departmentId
|
||||
- status
|
||||
- (requestId, departmentId)
|
||||
- (requestId, status)
|
||||
```
|
||||
|
||||
### Table: workflow_states
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
requestId: UUID (FK: license_requests) UNIQUE
|
||||
currentStageId: varchar(100)
|
||||
completedStages: jsonb (array of stage IDs, default: [])
|
||||
pendingApprovals: jsonb (array of {departmentCode, status, createdAt}, default: [])
|
||||
executionLog: jsonb (array of {timestamp, stageId, action, details}, default: [])
|
||||
stageStartedAt: TIMESTAMP WITH TIME ZONE NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- requestId
|
||||
```
|
||||
|
||||
### Table: webhooks
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
departmentId: UUID (FK: departments)
|
||||
url: text
|
||||
events: jsonb (array of event types, default: [])
|
||||
secretHash: varchar(255)
|
||||
isActive: boolean (default: true)
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
updatedAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- departmentId
|
||||
- (departmentId, isActive)
|
||||
```
|
||||
|
||||
### Table: webhook_logs
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
webhookId: UUID (FK: webhooks)
|
||||
eventType: varchar(100)
|
||||
payload: jsonb
|
||||
responseStatus: integer NULLABLE
|
||||
responseBody: text NULLABLE
|
||||
responseTime: integer NULLABLE [milliseconds]
|
||||
retryCount: integer (default: 0)
|
||||
status: ENUM (PENDING, SUCCESS, FAILED)
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- webhookId
|
||||
- eventType
|
||||
- status
|
||||
- createdAt
|
||||
- (webhookId, status)
|
||||
```
|
||||
|
||||
### Table: audit_logs
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
entityType: ENUM (REQUEST, APPROVAL, DOCUMENT, DEPARTMENT, WORKFLOW)
|
||||
entityId: UUID
|
||||
action: varchar(100)
|
||||
actorType: ENUM (APPLICANT, DEPARTMENT, SYSTEM, ADMIN)
|
||||
actorId: UUID NULLABLE
|
||||
oldValue: jsonb NULLABLE
|
||||
newValue: jsonb NULLABLE
|
||||
ipAddress: varchar(45) NULLABLE
|
||||
userAgent: text NULLABLE
|
||||
correlationId: varchar(255) NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
|
||||
Indexes:
|
||||
- entityType
|
||||
- entityId
|
||||
- action
|
||||
- actorType
|
||||
- createdAt
|
||||
- (entityType, entityId)
|
||||
- (actorId, createdAt)
|
||||
```
|
||||
|
||||
### Table: blockchain_transactions
|
||||
|
||||
```typescript
|
||||
id: UUID (PK)
|
||||
txHash: varchar(255) UNIQUE
|
||||
txType: ENUM (MINT_NFT, APPROVAL, DOC_UPDATE, REJECT, REVOKE)
|
||||
relatedEntityType: varchar(100)
|
||||
relatedEntityId: UUID
|
||||
fromAddress: varchar(255)
|
||||
toAddress: varchar(255) NULLABLE
|
||||
status: ENUM (PENDING, CONFIRMED, FAILED)
|
||||
blockNumber: bigint NULLABLE
|
||||
gasUsed: bigint NULLABLE
|
||||
errorMessage: text NULLABLE
|
||||
createdAt: TIMESTAMP WITH TIME ZONE
|
||||
confirmedAt: TIMESTAMP WITH TIME ZONE NULLABLE
|
||||
|
||||
Indexes:
|
||||
- txHash
|
||||
- status
|
||||
- txType
|
||||
- relatedEntityId
|
||||
- createdAt
|
||||
- (status, txType)
|
||||
```
|
||||
|
||||
## Enums Defined
|
||||
|
||||
### LicenseRequestStatus (license_requests table)
|
||||
- DRAFT
|
||||
- SUBMITTED
|
||||
- IN_REVIEW
|
||||
- PENDING_RESUBMISSION
|
||||
- APPROVED
|
||||
- REJECTED
|
||||
- REVOKED
|
||||
- CANCELLED
|
||||
|
||||
### ApprovalStatus (approvals table)
|
||||
- PENDING
|
||||
- APPROVED
|
||||
- REJECTED
|
||||
- CHANGES_REQUESTED
|
||||
- REVIEW_REQUIRED
|
||||
|
||||
### WebhookLogStatus (webhook_logs table)
|
||||
- PENDING
|
||||
- SUCCESS
|
||||
- FAILED
|
||||
|
||||
### AuditEntityType (audit_logs table)
|
||||
- REQUEST
|
||||
- APPROVAL
|
||||
- DOCUMENT
|
||||
- DEPARTMENT
|
||||
- WORKFLOW
|
||||
|
||||
### AuditActorType (audit_logs table)
|
||||
- APPLICANT
|
||||
- DEPARTMENT
|
||||
- SYSTEM
|
||||
- ADMIN
|
||||
|
||||
### BlockchainTransactionType (blockchain_transactions table)
|
||||
- MINT_NFT
|
||||
- APPROVAL
|
||||
- DOC_UPDATE
|
||||
- REJECT
|
||||
- REVOKE
|
||||
|
||||
### BlockchainTransactionStatus (blockchain_transactions table)
|
||||
- PENDING
|
||||
- CONFIRMED
|
||||
- FAILED
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Data Integrity
|
||||
- All foreign keys with CASCADE DELETE
|
||||
- UNIQUE constraints on critical fields
|
||||
- NOT NULL constraints where required
|
||||
- CHECK constraints via TypeORM validation
|
||||
|
||||
### 2. Performance Optimization
|
||||
- 40+ indexes covering all query patterns
|
||||
- Composite indexes for common joins
|
||||
- JSONB columns for flexible metadata
|
||||
- Partitioning ready for large audit tables
|
||||
|
||||
### 3. Audit & Compliance
|
||||
- Complete audit_logs tracking all changes
|
||||
- Actor identification (who made the change)
|
||||
- Old/new values for change comparison
|
||||
- Correlation IDs for distributed tracing
|
||||
- IP address and user agent capture
|
||||
|
||||
### 4. Blockchain Integration
|
||||
- blockchain_transactions table for on-chain tracking
|
||||
- NFT token ID field in license_requests
|
||||
- Transaction hash storage for verification
|
||||
- Block confirmation tracking
|
||||
|
||||
### 5. Workflow Management
|
||||
- workflow_states for execution tracking
|
||||
- Complete execution log with timestamps
|
||||
- Pending approvals tracking by department
|
||||
- Completed stages audit trail
|
||||
|
||||
### 6. Document Versioning
|
||||
- Multiple versions per document
|
||||
- SHA-256 hashing for integrity
|
||||
- File size and mime type tracking
|
||||
- Upload attribution and timestamps
|
||||
|
||||
### 7. Webhook System
|
||||
- Flexible event subscription model
|
||||
- Retry mechanism with count tracking
|
||||
- Response time monitoring
|
||||
- Status tracking (PENDING, SUCCESS, FAILED)
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### Prerequisites
|
||||
- PostgreSQL 12+ with UUID extension
|
||||
- Node.js 16+ with TypeORM
|
||||
- npm or yarn package manager
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Install Dependencies**
|
||||
```bash
|
||||
npm install typeorm pg uuid crypto dotenv
|
||||
```
|
||||
|
||||
2. **Create PostgreSQL Database**
|
||||
```bash
|
||||
createdb goa_gel_db
|
||||
```
|
||||
|
||||
3. **Configure Environment**
|
||||
Create `.env` file:
|
||||
```env
|
||||
DATABASE_HOST=localhost
|
||||
DATABASE_PORT=5432
|
||||
DATABASE_USER=postgres
|
||||
DATABASE_PASSWORD=your_password
|
||||
DATABASE_NAME=goa_gel_db
|
||||
DATABASE_LOGGING=true
|
||||
DATABASE_SSL=false
|
||||
NODE_ENV=development
|
||||
```
|
||||
|
||||
4. **Run Migrations**
|
||||
```bash
|
||||
npx typeorm migration:run -d src/database/data-source.ts
|
||||
```
|
||||
|
||||
5. **Seed Sample Data**
|
||||
```bash
|
||||
npx ts-node src/database/seeders/seed.ts
|
||||
```
|
||||
|
||||
6. **Verify Setup**
|
||||
```bash
|
||||
psql goa_gel_db -c "\dt"
|
||||
```
|
||||
|
||||
## Entity Relationships
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Applicant │
|
||||
└──────┬──────┘
|
||||
│ 1:N
|
||||
│
|
||||
├─────────────────────────────┬──────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────────┐ ┌──────────────┐ ┌────────────────┐
|
||||
│ LicenseRequest │ │ Workflow │ │ WorkflowState │
|
||||
└────┬─────────────┘ └──────────────┘ └────────────────┘
|
||||
│ △
|
||||
│ 1:N │
|
||||
├─────────┬──────────┐ │
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ 1:1 relation
|
||||
┌────────────┐ ┌───────────┐ ┌──────────────┐
|
||||
│ Document │ │ Approval │ │ Approval │
|
||||
│ 1:N │ │ 1:N │ │ Status │
|
||||
│ DocumentV │ │ Department│ │ Tracking │
|
||||
└────────────┘ └─────┬─────┘ └──────────────┘
|
||||
│
|
||||
│ N:1
|
||||
│
|
||||
┌──────▼──────┐
|
||||
│ Department │
|
||||
│ 1:N │
|
||||
│ Webhook │
|
||||
│ 1:N │
|
||||
│ WebhookLog │
|
||||
└─────────────┘
|
||||
|
||||
AuditLog ─── Tracks all changes to above entities
|
||||
|
||||
BlockchainTransaction ─── Records NFT minting and approvals
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Create a New Migration
|
||||
```bash
|
||||
npx typeorm migration:generate -d src/database/data-source.ts -n AddNewField
|
||||
```
|
||||
|
||||
### Generate Migration from Entity Changes
|
||||
```bash
|
||||
npx typeorm migration:generate -d src/database/data-source.ts -n AutoGenerated
|
||||
```
|
||||
|
||||
### Revert Last Migration
|
||||
```bash
|
||||
npx typeorm migration:revert -d src/database/data-source.ts
|
||||
```
|
||||
|
||||
### View Migration Status
|
||||
```bash
|
||||
npx typeorm migration:show -d src/database/data-source.ts
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **Hash Storage** - API keys, secrets, and webhook secrets are hashed with SHA-256
|
||||
2. **Wallet Addresses** - Normalized to lowercase to prevent duplication
|
||||
3. **Cascade Delete** - Foreign keys cascade to prevent orphaned records
|
||||
4. **Audit Trail** - All critical operations logged with actor identification
|
||||
5. **Correlation IDs** - Support distributed tracing for audit
|
||||
6. **JSONB Validation** - Additional validation at application layer
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Index Usage** - All frequently queried columns are indexed
|
||||
2. **Composite Indexes** - Multi-column queries optimized
|
||||
3. **JSONB Queries** - Use PostgreSQL native JSONB operations
|
||||
4. **Batch Operations** - Use chunking for large inserts
|
||||
5. **Connection Pooling** - Configured at 20 connections (production)
|
||||
|
||||
## File Locations
|
||||
|
||||
```
|
||||
/sessions/cool-elegant-faraday/mnt/Goa-GEL/backend/src/database/
|
||||
├── entities/
|
||||
│ ├── applicant.entity.ts
|
||||
│ ├── approval.entity.ts
|
||||
│ ├── audit-log.entity.ts
|
||||
│ ├── blockchain-transaction.entity.ts
|
||||
│ ├── department.entity.ts
|
||||
│ ├── document-version.entity.ts
|
||||
│ ├── document.entity.ts
|
||||
│ ├── index.ts
|
||||
│ ├── license-request.entity.ts
|
||||
│ ├── webhook-log.entity.ts
|
||||
│ ├── webhook.entity.ts
|
||||
│ ├── workflow-state.entity.ts
|
||||
│ └── workflow.entity.ts
|
||||
├── migrations/
|
||||
│ └── 1704067200000-InitialSchema.ts
|
||||
├── seeders/
|
||||
│ └── seed.ts
|
||||
├── data-source.ts
|
||||
├── index.ts
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Install PostgreSQL and create database
|
||||
2. Configure environment variables in `.env`
|
||||
3. Run migrations: `npx typeorm migration:run -d src/database/data-source.ts`
|
||||
4. Seed sample data: `npx ts-node src/database/seeders/seed.ts`
|
||||
5. Start backend application
|
||||
6. Verify database tables: `psql goa_gel_db -c "\dt"`
|
||||
|
||||
## Support
|
||||
|
||||
For detailed information, see:
|
||||
- `/src/database/README.md` - Database documentation
|
||||
- `/src/database/entities/` - Entity definitions with comments
|
||||
- `/src/database/migrations/` - SQL migration details
|
||||
Reference in New Issue
Block a user