YAML Import/Export Documentation

This document explains how to use the YAML import/export functionality to backup and restore your Korgraph database.

Overview

Korgraph provides comprehensive YAML-based import/export functionality that allows you to:

• Export your entire database to human-readable YAML files
• Import YAML files to restore or migrate your architecture
• Separate layout positions from entity data for better organization
• Start fresh with a new database from YAML files

Export Functionality

Export Database (Entities)

Export all entities (companies, domains, contexts, systems, etc.) excluding layout positions:

# Using curl
curl http://localhost:3000/api/export/yaml > korgraph-export.yaml

# Using wget
wget http://localhost:3000/api/export/yaml -O korgraph-export.yaml

What's included:

• Companies, domains, subdomains, contexts
• Systems, system modules, system landscapes
• Processes, process landscapes
• Actors, interfaces, events, data objects
• Business products, regions, teams
• Notes and features

What's excluded:

• Layout positions (exported separately)
• Design documents (CouchDB internal)

Export Layout Positions

Export all diagram layout positions separately:

# Using curl
curl http://localhost:3000/api/export/yaml-layouts > korgraph-layouts.yaml

# Using wget
wget http://localhost:3000/api/export/yaml-layouts -O korgraph-layouts.yaml

What's included:

• Element positions (x, y coordinates) for each diagram
• Diagram IDs (company-1, domain-xyz, etc.)

Import Functionality

Import Database (Entities)

Import entities from a YAML file:

curl -X POST \
  http://localhost:3000/api/import/yaml \
  -H "Content-Type: application/x-yaml" \
  --data-binary @korgraph-export.yaml

Behavior:

• Creates new elements if they don't exist
• Updates existing elements (matches by ID)
• Skips invalid elements and reports errors
• Returns import statistics (created, updated, failed)

Response Example:

{
  "success": true,
  "message": "Database import completed",
  "stats": {
    "total": 150,
    "created": 120,
    "updated": 30,
    "failed": 0,
    "errors": []
  }
}

Import Layout Positions

Import layout positions from a YAML file:

curl -X POST \
  http://localhost:3000/api/import/yaml-layouts \
  -H "Content-Type: application/x-yaml" \
  --data-binary @korgraph-layouts.yaml

Behavior:

• Creates new layout documents if they don't exist
• Updates existing layouts (matches by diagram ID)
• Preserves element positions exactly as specified

Complete Workflow Examples

Full Backup

Create a complete backup of your database:

#!/bin/bash
DATE=$(date +%Y-%m-%d)
BACKUP_DIR="./backups/$DATE"

mkdir -p "$BACKUP_DIR"

# Export entities
curl http://localhost:3000/api/export/yaml > "$BACKUP_DIR/korgraph-export.yaml"

# Export layouts
curl http://localhost:3000/api/export/yaml-layouts > "$BACKUP_DIR/korgraph-layouts.yaml"

echo "Backup created in $BACKUP_DIR"

Full Restore

Restore from a backup:

#!/bin/bash
BACKUP_DIR="./backups/2025-10-18"

# Import entities
curl -X POST \
  http://localhost:3000/api/import/yaml \
  -H "Content-Type: application/x-yaml" \
  --data-binary "@$BACKUP_DIR/korgraph-export.yaml"

# Import layouts
curl -X POST \
  http://localhost:3000/api/import/yaml-layouts \
  -H "Content-Type: application/x-yaml" \
  --data-binary "@$BACKUP_DIR/korgraph-layouts.yaml"

echo "Restore completed"

Start from Scratch

Initialize a fresh database with predefined data:

#!/bin/bash

# 1. Stop the server
npm stop

# 2. Delete existing databases (optional - only if you want to start completely fresh)
rm -rf data/db/korgraph
rm -rf data/db/korgraph_layouts

# 3. Start the server (will create empty databases)
npm start

# Wait for server to start
sleep 3

# 4. Import your YAML files
curl -X POST \
  http://localhost:3000/api/import/yaml \
  -H "Content-Type: application/x-yaml" \
  --data-binary @./my-architecture.yaml

curl -X POST \
  http://localhost:3000/api/import/yaml-layouts \
  -H "Content-Type: application/x-yaml" \
  --data-binary @./my-layouts.yaml

echo "Fresh database initialized!"

YAML File Format

Database Export Format

# Korgraph Database Export
# Generated: 2025-10-18T10:30:00.000Z
# Version: 1.0
# Note: Layout positions are exported separately to korgraph-layouts.yaml

export:
  date: "2025-10-18T10:30:00.000Z"
  version: "1.0"
  elementCount: 150

data:
  company:
    - id: "company-001"
      rev: "1-abc123"
      name: "Example Corporation"
      description: "Main company entity"
      elementType: "company"

  domain:
    - id: "domain-001"
      rev: "1-def456"
      name: "Sales Domain"
      description: "Handles all sales processes"
      elementType: "domain"
      companyId: "company-001"
      
  # ... more element types

Layouts Export Format

# Korgraph Layout Positions Export
# Generated: 2025-10-18T10:30:00.000Z
# Version: 1.0

export:
  date: "2025-10-18T10:30:00.000Z"
  version: "1.0"
  layoutCount: 25

layouts:
  - diagramId: "company-001"
    rev: "1-xyz789"
    elements:
      - elementId: "domain-001"
        x: 100
        y: 200
      - elementId: "domain-002"
        x: 400
        y: 200

  - diagramId: "domain-001"
    rev: "1-abc123"
    elements:
      - elementId: "subdomain-001"
        x: 50
        y: 100
      - elementId: "subdomain-002"
        x: 250
        y: 100

Migration Strategy

When migrating between environments:

1. Export from source:

   curl http://source-server:3000/api/export/yaml > export.yaml
   curl http://source-server:3000/api/export/yaml-layouts > layouts.yaml

2. Transfer files to target environment

3. Import to target:

   curl -X POST http://target-server:3000/api/import/yaml \
     -H "Content-Type: application/x-yaml" --data-binary @export.yaml
   
   curl -X POST http://target-server:3000/api/import/yaml-layouts \
     -H "Content-Type: application/x-yaml" --data-binary @layouts.yaml

Error Handling

Import Errors

If import fails for some elements, the API returns details:

{
  "success": true,
  "message": "Database import completed",
  "stats": {
    "total": 150,
    "created": 145,
    "updated": 0,
    "failed": 5,
    "errors": [
      {
        "id": "invalid-element-1",
        "type": "company",
        "error": "Missing required field: name"
      }
    ]
  }
}

Handling Conflicts

The import process automatically handles conflicts:

• If element exists: updates with new data
• If element doesn't exist: creates new element
• CouchDB revisions are handled automatically

Best Practices

1. Regular Backups: Set up automated daily/weekly backups
2. Version Control: Store YAML files in Git for version history
3. Separate Concerns: Keep entities and layouts in separate files
4. Test Imports: Always test imports in a development environment first
5. Incremental Updates: For large datasets, consider importing in batches

Limitations

• File Size: Maximum 50MB per import (configurable in server.js)
• Performance: Large imports may take several minutes
• Validation: Import validates basic YAML structure but not all business rules
• Revisions: Exported _rev fields are not imported (CouchDB assigns new revisions)

Troubleshooting

Import Returns 500 Error

Check server logs for detailed error messages:

npm start | grep "YAML Import"

Invalid YAML Format

Ensure your YAML follows the exact format shown above. Common issues:

• Incorrect indentation (must use 2 spaces)
• Missing required fields (id, elementType)
• Invalid YAML syntax

Database Connection Issues

Ensure CouchDB is running and accessible:

curl http://localhost:5984

Security Considerations

• Authentication: Add authentication middleware before deploying to production
• File Validation: The import validates structure but accepts any valid YAML
• Backup Security: Store backups in a secure location
• Access Control: Limit who can perform imports/exports

Future Enhancements

Potential improvements:

• Incremental import (only changed elements)
• Import validation with detailed error reports
• Compressed export formats
• Automatic backup scheduling
• Web UI for import/export operations