What is Temporal?

Created:

Temporal is an architecture and way of running code that allows you to run long-running processes in a deterministic, predictable way (amongst other things).

To understand what that means, let's delve into a problem that Temporal could help solve: how could we effectively execute a process for 30 minutes and keep the user informed in the frontend? To make it more concrete, envision a scenario where the user creates a CDN distribution that takes 30 minutes to setup through a webpage that will inform them about the creation status.

A possible solution would be to implement an API server and a Worker server that can communicate with each other. The Worker server would be responsible for handling long-running processes and updating the relevant database, while the frontend would retrieve data from the database to keep the user updated on the progress of the creation process.

Here's a contrived, high-level pseudo-code example of what the backend part could look like:

// api.js
import {worker} from './worker' // Worker client that can do long-running processes

// API Server
server.route('/createWebsite', async () => {
   const id = worker.start('create-cloudfront')
   return {
	   id
   };
})
server.route('/getWebsite', async(request) => {
	const id = request
	const website = db.website.find({ id })
	return {
	    ...website
		status: website.status
	}
})

This could work but there's a lot of complexity we would need to handle, to name a few:

  • What happens if the worker server crashes mid-way through the execution? How can we recover from that?
  • What happens if the worker receives 1000 different requests that last 30 minutes?
  • What happens if we need to support the user canceling the creation mid-way?
  • What if now we also want to inform the user via email once the process completes?

This and other challenges are what temporal attempts to solve.

How Does Temporal Work?

Similar to our pseudo-code above, the top-level architecture in Temporal has 2 elements:

  • The temporal server, called the "Temporal Cluster"
    • This can be managed by Temporal with their cloud solution. This server is what will orchestrate your workers.
  • The workers, called "Worker Processes"
    • You manage this. It's where your application code will be executed.
    • The worker process is also called "Workflow Executions"

Within the workers, the 2 main concepts you'll need to become familiar with to develop your application are Workflows & Activities.

Temporal Workflow

A Workflow is the "main unit of execution" of your application. There's a lot of complexity that goes into it, but as an app developer the most relevant parts are:

  • In practical terms, a Workflow is an async function that will be executed within the Temporal Worker (which is triggered from the Temporal Cluster).
  • It needs to be deterministic and because of that the code used for a Temporal Workflow will have a bunch of limitations:
    • Whereas in a normal JS/TS file you can import functions from other files and execute them, in a workflow file you can only import functions that have pure JS/TS and no third-party interactions.
      • This means you cannot directly import ANY file that has side effects like calling an API or updating the database - that's where the Activities come in!
    • Because of the above, most of your imported functions will be Activities and you need to be extra careful when importing code in another way as the build will break if you import an NPM module that is not compatible.
    • Functions that have non-deterministic behavior, like Math.random, cannot be used.
  • As mentioned above, functions that use third-party libraries or services need to be imported as Activities. The import is done through a Temporal "proxy" (proxyActivities).
  • A Workflow can be canceled and you can handle how you finish the workflow once it has received a cancellation.
  • You can trigger Child Workflows
    • You can either execute or start the child workflow
      • execute waits for the workflow to finish
      • start is a "fire & forget" sort of thing
  • Every Workflow has an ID associated with it, and there can only ever be 1 workflow with that ID running at the same time
    • Once it finishes, we can start a workflow with that ID again and it will have a different workflow run ID

Here's a simplified example of what a workflow looks like:

import { proxyActivities } from '@temporalio/workflow';
import type * as activities from './activities';

const { sendEmail } = proxyActivities<typeof activities>({
	retry: {
		maximumAttempts: 2,
	},
	startToCloseTimeout: '30 seconds',
});


export async function sendWelcomeEmailWorkflow(email: string) {
	try {
		await sendEmail({ recipient: email, type: "welcome" });
			return {
				success: true,
			}
		} catch (err) {
			return {
				success: false,
			}
	}
}

Temporal Activity

An Activity is a lot easier to understand compared to a Workflow. It's a normal function that may contain non-deterministic code.

Some aspects that make them different than normal functions are related to the fact that they run inside a Workflow, namely:

  • Depending on the configuration when imported in a Workflow (in the proxyActivities options), an Activity can potentially be retried. This means you can try/catch errors inside the activity and throw a "retryable" or "nonRetryable" error.
  • Besides retries, there are a couple of more options that can be configured when they're imported, such as timeout or how they should behave on cancellation.
  • Due to how temporal orchestrates it all, the input and output for the Activity functions cannot be complex types like a class or another function and if given, they will be simplified to objects that can be stringified.
    • As an example, if you pass an Error type to an Activity from a Workflow, it will be stringified to an object and it will no longer be error instanceof Error.
    • The exception to this is if you import the function inside the Activity. In this scenario, the imported function would behave just like a normal function without any Temporal limitations.

Temporal Client

The last essential piece to understand is how the Workflows can be triggered from a frontend app for example. This can be done using the Temporal Client.

Here's an example of a Temporal Client calling a workflow:

import { Connection, Client as TemporalClient } from '@temporalio/client'
import { example } from './workflows'

const connection = await Connection.connect()
const client = new TemporalClient({
  connection,
})

const handle = await client.workflow.start('sendWelcomeEmailWorkflow', {
  args: ['[email protected]'],
  taskQueue: 'hello-world',
  workflowId: 'workflow-with-meaningful-id',
})

Other Advanced Concepts

The concepts mentioned above are the basics to get started with Temporal, but there are more advanced concepts that I didn't cover such as Interceptors, Signals, and Scheduled Workflow, to name a few.

Reference