What is Temporal?

Temporal is a workflow orchestration platform that enables developers to build reliable, long-running applications that maintain their state even when facing system failures or interruptions. It provides a robust framework for implementing complex business processes that might take minutes, hours, or even days to complete.

The Problem Temporal Solves

Imagine this scenario: You're building a web application where users can create CDN distributions, but the process takes 30 minutes to complete. How would you:

Keep the process running reliably for the entire duration?
Maintain state if servers crash or restart?
Communicate progress to the user throughout the process?
Allow users to cancel operations mid-process?
Send notifications when the process completes?

Traditional approaches often involve complex setups with separate worker servers, queue management, state tracking, and error handling. Let's look at how this might be implemented without Temporal:

// api.js - Traditional approach
import { worker } from './worker'; // Worker client for long-running processes
import db from './database';

// API Server
server.route('/createWebsite', async (request, response) => {
   try {
      // Generate a unique ID for this job
      const jobId = generateUniqueId();

      // Store initial state in database
      await db.website.create({
         id: jobId,
         status: 'PENDING',
         createdAt: new Date()
      });

      // Queue the job to be processed by workers
      worker.enqueue('create-cloudfront', {
         jobId,
         userId: request.user.id,
         domainName: request.body.domainName
      });

      return response.json({ id: jobId });
   } catch (error) {
      console.error('Failed to start website creation:', error);
      return response.status(500).json({ error: 'Failed to start process' });
   }
});

server.route('/getWebsite/:id', async (request, response) => {
   try {
      const website = await db.website.findOne({ id: request.params.id });

      if (!website) {
         return response.status(404).json({ error: 'Website not found' });
      }

      return response.json({
         id: website.id,
         status: website.status,
         url: website.url,
         createdAt: website.createdAt,
         completedAt: website.completedAt
      });
   } catch (error) {
      console.error('Failed to fetch website:', error);
      return response.status(500).json({ error: 'Failed to retrieve website' });
   }
});

While this approach can work, it comes with significant challenges:

Reliability: What happens if the worker crashes mid-execution?
Scalability: How do you handle many concurrent long-running processes?
State Management: How do you persist and recover workflow state?
Cancellation: How do you gracefully handle user cancellations?
Monitoring: How do you track progress and send notifications?

Temporal is designed specifically to address these challenges.

How Does Temporal Work?

Temporal's architecture consists of two primary components:

Temporal Cluster (Server):
- Orchestrates and tracks the state of all workflows
- Handles scheduling, retries, and timeouts
- Can be self-hosted or managed via Temporal Cloud
Worker Processes:
- Run your application code
- Connect to the Temporal Cluster
- Execute Workflows and Activities

Temporal introduces two key concepts to structure your application: Workflows and Activities.

Temporal Workflows

A Workflow is the core organizing unit in Temporal. It defines the sequence of steps and the overall business logic of your process.

Key characteristics of Workflows:

Durable: Workflows maintain their state across server failures and restarts
Deterministic: Workflows must produce the same outputs given the same inputs
Long-Running: Workflows can run for seconds to years
Versioned: Workflows support deployment of new versions without affecting running instances

Here's a simple Workflow example for our CDN creation scenario:

// workflows.ts
import { proxyActivities } from '@temporalio/workflow';
import type * as activities from './activities';

// Import activities with configuration
const {
  createCloudFrontDistribution,
  configureSSL,
  updateDNS,
  sendCompletionEmail
} = proxyActivities<typeof activities>({
  retry: { maximumAttempts: 3 },
  startToCloseTimeout: '30 minutes'
});

// The main workflow function
export async function createWebsiteWorkflow(params: {
  domainName: string;
  userId: string;
  email: string;
}) {
  // These steps will be executed in sequence, and state is preserved
  // even if the worker crashes between steps

  // Step 1: Create CloudFront distribution
  const distribution = await createCloudFrontDistribution(params.domainName);

  // Step 2: Configure SSL certificate
  const certificate = await configureSSL(distribution.id, params.domainName);

  // Step 3: Update DNS records
  await updateDNS(params.domainName, distribution.domainName);

  // Step 4: Send completion notification
  await sendCompletionEmail({
    email: params.email,
    websiteUrl: `https://${params.domainName}`,
    distributionId: distribution.id
  });

  // Return the result
  return {
    success: true,
    distributionId: distribution.id,
    websiteUrl: `https://${params.domainName}`
  };
}

This workflow has several advantages over the traditional approach:

State Persistence: If the worker crashes at any point, it will automatically resume from the last completed step
Error Handling: Failed activities will be retried automatically based on retry policies
Visibility: Temporal tracks the progress and state of each workflow
Simplicity: The workflow reads like a synchronous script, despite executing over a long period

Temporal Activities

An Activity is a single task or step within a workflow. Unlike workflows, activities:

Can be Non-Deterministic: Activities can call external services, use random numbers, etc.
Are Independent: Each activity is executed as a separate unit
Can Have Different Characteristics: Each activity can have its own timeout, retry policy, etc.

Here's an example of activities for our CDN creation scenario:

// activities.ts
import { CloudFrontClient, CreateDistributionCommand } from '@aws-sdk/client-cloudfront';
import { CertificateManager } from '@aws-sdk/client-acm';
import { Route53Client, ChangeResourceRecordSetsCommand } from '@aws-sdk/client-route-53';
import { sendEmail } from './email-service';
import db from './database';

// Initialize AWS clients
const cloudfront = new CloudFrontClient({ region: 'us-east-1' });
const acm = new CertificateManager({ region: 'us-east-1' });
const route53 = new Route53Client({ region: 'us-east-1' });

// Activity to create CloudFront distribution
export async function createCloudFrontDistribution(domainName: string) {
  console.log(`Creating CloudFront distribution for ${domainName}`);

  // Update progress in database
  await db.website.updateOne(
    { domainName },
    { $set: { status: 'CREATING_DISTRIBUTION' }}
  );

  // Create the distribution (actual AWS SDK call)
  const response = await cloudfront.send(new CreateDistributionCommand({
    // CloudFront configuration parameters
    DistributionConfig: {
      CallerReference: `${domainName}-${Date.now()}`,
      Origins: {/* ... */},
      DefaultCacheBehavior: {/* ... */},
      // Other required parameters
    }
  }));

  const distributionId = response.Distribution?.Id;
  const distributionDomain = response.Distribution?.DomainName;

  // Save distribution details to database
  await db.website.updateOne(
    { domainName },
    {
      $set: {
        distributionId,
        distributionDomain,
        status: 'DISTRIBUTION_CREATED'
      }
    }
  );

  return {
    id: distributionId,
    domainName: distributionDomain
  };
}

// Activity to configure SSL
export async function configureSSL(distributionId: string, domainName: string) {
  console.log(`Configuring SSL for ${domainName}`);

  // Update progress
  await db.website.updateOne(
    { domainName },
    { $set: { status: 'CONFIGURING_SSL' }}
  );

  // Request and validate certificate (AWS SDK calls)
  // ... SSL configuration code

  return { certificateArn };
}

// Activity to update DNS
export async function updateDNS(domainName: string, distributionDomain: string) {
  console.log(`Updating DNS records for ${domainName}`);

  // Update progress
  await db.website.updateOne(
    { domainName },
    { $set: { status: 'UPDATING_DNS' }}
  );

  // Add CNAME record to Route53 (AWS SDK call)
  // ... DNS configuration code

  // Mark as complete in database
  await db.website.updateOne(
    { domainName },
    { $set: { status: 'COMPLETED', completedAt: new Date() }}
  );

  return true;
}

// Activity to send completion email
export async function sendCompletionEmail(params: {
  email: string;
  websiteUrl: string;
  distributionId: string;
}) {
  console.log(`Sending completion email to ${params.email}`);

  // Send email using email service
  await sendEmail({
    to: params.email,
    subject: 'Your website is ready!',
    body: `Your website at ${params.websiteUrl} has been successfully deployed.`,
  });

  return true;
}

Initiating Workflows with the Temporal Client

To start a workflow from your API server, you use the Temporal Client:

// api.js - Temporal approach
import { Connection, Client } from '@temporalio/client';
import express from 'express';
import db from './database';

const app = express();
app.use(express.json());

// Initialize Temporal client
let temporalClient;
async function initTemporalClient() {
  const connection = await Connection.connect({ address: 'localhost:7233' });
  temporalClient = new Client({ connection });
}
initTemporalClient();

// API endpoint to create a website
app.post('/createWebsite', async (request, response) => {
  try {
    const { domainName } = request.body;
    const userId = request.user.id;
    const email = request.user.email;

    // Create a unique workflow ID
    const workflowId = `create-website-${domainName}-${Date.now()}`;

    // Create initial record in database
    await db.website.create({
      domainName,
      userId,
      status: 'PENDING',
      workflowId,
      createdAt: new Date()
    });

    // Start the Temporal workflow
    const handle = await temporalClient.workflow.start('createWebsiteWorkflow', {
      args: [{ domainName, userId, email }],
      taskQueue: 'website-creation',
      workflowId,
    });

    // Return the workflow ID to the client
    return response.json({
      id: workflowId,
      domainName,
      status: 'PENDING'
    });
  } catch (error) {
    console.error('Failed to start website creation:', error);
    return response.status(500).json({ error: 'Failed to start process' });
  }
});

// API endpoint to get website status
app.get('/website/:id', async (request, response) => {
  try {
    const website = await db.website.findOne({ workflowId: request.params.id });

    if (!website) {
      return response.status(404).json({ error: 'Website not found' });
    }

    return response.json({
      id: website.workflowId,
      domainName: website.domainName,
      status: website.status,
      createdAt: website.createdAt,
      completedAt: website.completedAt || null
    });
  } catch (error) {
    console.error('Failed to fetch website:', error);
    return response.status(500).json({ error: 'Failed to retrieve website' });
  }
});

// API endpoint to cancel website creation
app.delete('/website/:id', async (request, response) => {
  try {
    const website = await db.website.findOne({ workflowId: request.params.id });

    if (!website) {
      return response.status(404).json({ error: 'Website not found' });
    }

    // Cancel the workflow
    const handle = temporalClient.workflow.getHandle(request.params.id);
    await handle.cancel();

    // Update status in database
    await db.website.updateOne(
      { workflowId: request.params.id },
      { $set: { status: 'CANCELLED' }}
    );

    return response.json({ success: true });
  } catch (error) {
    console.error('Failed to cancel website creation:', error);
    return response.status(500).json({ error: 'Failed to cancel process' });
  }
});

app.listen(3000, () => console.log('Server running on port 3000'));

Benefits of Using Temporal

By using Temporal for our CDN creation process, we gain several advantages:

Durability: Workflows continue where they left off, even after system failures
Visibility: Built-in tools for monitoring and debugging workflow executions
Scalability: Easy to handle thousands of concurrent workflows
Cancellation: Simple built-in support for canceling workflows
Error Handling: Sophisticated retry policies and error handling
Versioning: Update workflow definitions without affecting running instances

Advanced Temporal Concepts

Beyond the basics, Temporal offers additional powerful features:

Signals: Send information to running workflows
Queries: Get the current state of a running workflow
Child Workflows: Create hierarchical workflow relationships
Scheduled Workflows: Schedule workflows to run at specific times
Interceptors: Add cross-cutting concerns like logging or metrics
Saga Pattern: Implement compensating transactions for rollbacks

Conclusion

Temporal provides a robust solution for implementing complex, long-running processes that need to be resilient to failures. By separating the orchestration logic (Workflows) from the implementation details (Activities), Temporal enables developers to build reliable distributed systems without having to reinvent all the infrastructure for durability, retries, and state management.

For developers building systems that need to coordinate multiple services over extended periods, Temporal offers a compelling alternative to traditional approaches with significantly less boilerplate code and simpler error handling.

References

Temporal Concepts: https://docs.temporal.io/concepts
Temporal TypeScript SDK: https://docs.temporal.io/develop/typescript
Temporal TypeScript Samples: https://github.com/temporalio/samples-typescript