nebula.js/docs/spec-command-testing.md

Spec Command Testing Setup

Overview

This document describes the testing infrastructure for the nebula spec command, which generates JSON schemas and TypeScript defaults from TypeScript interface definitions.

Test Strategy

The spec command is tested as part of the test-create CI job, which validates that:

1. The TypeScript template can be created successfully
2. The spec command runs without errors
3. Generated files have the correct structure and content
4. The generated files are valid TypeScript/JSON

Template Structure

TypeScript Template (commands/create/templates/sn/typescript/)

The TypeScript template includes:

• src/PropertyDef.ts - TypeScript interface with JSDoc @default annotations
• nebula.config.js - Configuration for the spec command
• tsconfig.json - TypeScript compiler configuration
• _package.json - Includes ts-json-schema-generator and TypeScript dependencies

Key Files

PropertyDef.ts

Defines chart properties with JSDoc annotations:

export interface ChartProperties {
  /**
   * Chart title
   * @default ""
   */
  title?: string;
}

nebula.config.js

Configures the spec command:

module.exports = {
  spec: {
    input: 'src/PropertyDef.ts',
    output: 'generated',
    interface: 'ChartProperties',
  },
};

Test Flow

1. Template Creation

The test script creates a TypeScript visualization project:

.github/scripts/nebula_create.sh generated/typescript-chart none false false true true true true

Parameters:

• PROJECT_NAME: generated/typescript-chart
• PICASSO_TEMPLATE: none
• MASHUP: false
• INSTALL: false
• BUILD: true
• TEST: true
• TYPESCRIPT: true (enables TypeScript template)
• TEST_SPEC: true (enables spec command testing)

2. Spec Command Execution

The script runs:

cd generated/typescript-chart
yarn run spec

3. Validation

The test validates:

File Creation

• generated/generated-default-properties.ts exists
• generated/*-properties.schema.json exists (name derived from package.json)

Schema Structure

• Contains $id field
• Contains definitions object
• Valid JSON format

Defaults Structure

• Contains TypeScript const declaration
• Proper type annotations
• Imports from PropertyDef

4. CI Artifacts

Generated files are uploaded as artifacts:

- name: Store TypeScript spec generated files
  uses: actions/upload-artifact@v5
  with:
    name: typescript-spec-generated
    path: generated/typescript-chart/generated

Test Script (nebula_create.sh)

Spec-Specific Logic

if [ "$TEST_SPEC" = "true" ]; then
  echo "Running spec command..."
  yarn run spec

  # Verify generated files exist
  if [ ! -f "generated/generated-default-properties.ts" ]; then
    echo "ERROR: generated-default-properties.ts was not created"
    exit 1
  fi

  # Find schema file dynamically
  SCHEMA_FILE=$(find generated -name "*-properties.schema.json" | head -n 1)
  if [ -z "$SCHEMA_FILE" ]; then
    echo "ERROR: JSON schema file was not created"
    exit 1
  fi

  # Validate schema structure
  if ! grep -q '"$id"' "$SCHEMA_FILE"; then
    echo "ERROR: Schema file missing $id field"
    exit 1
  fi

  # Validate defaults structure
  if ! grep -q "const.*Defaults.*:" "generated/generated-default-properties.ts"; then
    echo "ERROR: Defaults file has incorrect structure"
    exit 1
  fi
fi

CI Workflow

Test-Create Job

- name: Create Nebula TypeScript project with spec command test
  run: .github/scripts/nebula_create.sh generated/typescript-chart none false false true true true true

This step:

1. Creates a TypeScript template project
2. Links local @nebula.js packages
3. Runs the spec command
4. Validates generated files
5. Builds the project
6. Runs e2e tests

Extending the Tests

Adding More Validations

To add more validation steps, edit .github/scripts/nebula_create.sh:

# Example: Verify specific default values
if ! grep -q '"showTitles": true' "generated/generated-default-properties.ts"; then
  echo "ERROR: Default value for showTitles not found"
  exit 1
fi

# Example: Validate schema definitions
if ! jq -e '.definitions.ChartProperties' "$SCHEMA_FILE" > /dev/null; then
  echo "ERROR: ChartProperties definition not found in schema"
  exit 1
fi

Testing Different Configurations

Create additional test runs with different configurations:

- name: Test spec with custom config
  run: |
    cd generated/typescript-chart
    echo 'module.exports = { spec: { schemaOnly: true } };' > nebula.config.js
    yarn run spec
    # Validate only schema was generated

Local Testing

To test locally:

# Build the monorepo
yarn build

# Run the test script
.github/scripts/nebula_create.sh test-project none false false true true true true

# Check generated files
ls -la test-project/generated/
cat test-project/generated/generated-default-properties.ts

Troubleshooting

Common Issues

1. Missing dependencies: Ensure ts-json-schema-generator is installed
2. Invalid TypeScript: Check PropertyDef.ts syntax
3. Schema generation fails: Verify tsconfig.json is valid
4. Defaults not extracted: Ensure @default JSDoc tags are present

Debug Output

Add debugging to the test script:

if [ "$TEST_SPEC" = "true" ]; then
  echo "=== Debug: Running spec command ==="
  yarn run spec --verbose

  echo "=== Debug: Generated files ==="
  ls -lah generated/

  echo "=== Debug: Schema content ==="
  cat generated/*-properties.schema.json | jq '.'
fi

Future Enhancements

Potential improvements to the testing setup:

1. Unit tests: Add Jest tests for the spec command itself
2. Snapshot testing: Compare generated files against snapshots
3. Multiple templates: Test with different TypeScript configurations
4. Integration tests: Verify generated files work in actual charts
5. Performance testing: Measure generation time for large interfaces