Concurrency
Concurrency means doing several things in overlapping time instead of one after the other. A dashboard that needs five numbers can ask for all five at once and wait about as long as the slowest one, instead of the sum of all five.
In a Strutvane app there are three tools for this, and picking the right one matters more than anything else on this page:
| You want to… | Use | Built? |
|---|---|---|
| wait for several database queries or HTTP calls at once | Promise.all() (plain JavaScript) |
yes, it’s part of the language |
| run CPU-heavy work (parsing a big file, crunching numbers) without freezing the app | Concurrency.run([task(…), …]) on worker threads or processes |
planned for 0.3 |
| do something after the response has been sent | defer(fn) |
planned for 0.2 |
| do something later, reliably, with retries | a queued job | planned for 0.3 |
| run another program | Process |
planned for 0.3 |
The smallest example
Section titled “The smallest example”A stats endpoint needs three counts. Ask for all three at once:
import type { HttpContext } from '@strutvane/framework'import { Post } from '../../models/post.ts'import { Tag } from '../../models/tag.ts'import { User } from '../../models/user.ts'
export class StatsController { async show(ctx: HttpContext): Promise<Response> { const [posts, users, tags] = await Promise.all([ Post.query().count(), User.query().count(), Tag.query().count(), ]) return ctx.json({ posts, users, tags }) }}What you’ll see:
{ "posts": 128, "users": 14, "tags": 9 }If each count takes 30 ms, the response takes about 30 ms instead of 90 ms. Promise.all() starts all three queries, then waits until every one has answered, and gives you the answers in the same order you listed them.
Why one thread is enough for waiting
Section titled “Why one thread is enough for waiting”Node runs your code on one thread (one line of execution). That sounds limiting, but most of a web request is waiting: for the database, for another API, for a file. While one query waits, Node is free to start the next one. That’s why Promise.all() gives real speed-ups for database and HTTP work without any threads.
The work this does not help with is CPU-bound work: code that keeps the processor busy (resizing a big image in JavaScript, parsing a 200 MB CSV, hashing thousands of values). While it runs, that thread can do nothing else, so every other request on the server waits. That’s what Concurrency.run() is for.
When some of them fail
Section titled “When some of them fail”Promise.all(): stop at the first failure
Section titled “Promise.all(): stop at the first failure”If one call fails, Promise.all() fails at once with that error. Use it when you need every answer:
import { Project } from '../models/project.ts'import { Task } from '../models/task.ts'
export async function projectSummary(projectId: string) { const [project, openTasks] = await Promise.all([ Project.findOrFail(projectId), // throws if missing Task.query().where('project_id', projectId).where('status', 'open').count(), ]) return { name: project.name, openTasks }}What you’ll see for an unknown project: the ModelNotFoundError from findOrFail, which your app turns into a 404, exactly as if you’d awaited the calls one by one. The other query still runs to the end in the background; its answer is thrown away.
Promise.allSettled(): keep what worked
Section titled “Promise.allSettled(): keep what worked”When each part is optional (a dashboard where one widget may fail), use Promise.allSettled(). It never fails; it tells you which parts worked:
import { Log } from '@strutvane/framework'import { Post } from '../models/post.ts'import { fetchPartnerStatus } from './partner.ts'
export async function dashboard() { const [posts, partner] = await Promise.allSettled([ Post.query().where('status', 'published').count(), fetchPartnerStatus(), // another company's API: may be down ]) if (partner.status === 'rejected') Log.warning('Partner status unavailable', { error: partner.reason }) return { published: posts.status === 'fulfilled' ? posts.value : null, partner: partner.status === 'fulfilled' ? partner.value : 'unavailable', }}What you’ll see when the partner API is down:
{ "published": 128, "partner": "unavailable" }The page still loads; only the partner widget shows “unavailable”.
Don’t start thousands at once
Section titled “Don’t start thousands at once”Promise.all() over a list of 10,000 items starts 10,000 queries at the same moment. Your database has a limited number of connections, so most of them queue up, and the database may refuse new ones. Work in chunks:
import { recalculateProject } from './recalculate-project.ts'
/** At most 10 projects at a time. */export async function recalculateAll(projectIds: readonly string[]): Promise<void> { for (let start = 0; start < projectIds.length; start += 10) { const chunk = projectIds.slice(start, start + 10) await Promise.all(chunk.map((id) => recalculateProject(id))) }}For really large amounts of work, push one queued job per item (or a batch) and let workers spread it out.
Inside a transaction, one at a time
Section titled “Inside a transaction, one at a time”A transaction runs on one database connection, and a connection runs one statement at a time. Inside transaction(), await each write in turn instead of using Promise.all(): the writes can’t overlap anyway, and keeping them in order makes failures easy to follow.
CPU-heavy work: Concurrency.run()
Section titled “CPU-heavy work: Concurrency.run()”Concurrency.run() takes a list of tasks, runs each one on another thread or process, and returns their results in order. The main thread stays free to answer other requests while they work.
A task for a worker is a module, not a closure. A closure (a function that uses variables from the code around it) can’t be copied to another thread, because the other thread has its own memory. So each task lives in its own file and is named by its path; you hand it plain arguments.
Step 1: write the task module.
/** Parses one chunk of an uploaded CSV. Runs on a worker: plain data in, plain data out. */export default async function parseCsvChunk(args: { csv: string }): Promise<Array<{ title: string; status: string }>> { return args.csv .split('\n') .filter((line) => line.trim() !== '') .map((line, index) => { const [title, status] = line.split(',') if (title === undefined || status === undefined) throw new Error(`Row ${index + 1}: expected 2 columns`) return { title, status } })}Step 2: run it four times at once.
import { Concurrency, task } from '@strutvane/framework'
/** Parses a large upload in four parts at once, on other threads. */export async function previewImport(chunks: readonly [string, string, string, string]) { const [a, b, c, d] = await Concurrency.run([ task('imports/parse-csv-chunk', { csv: chunks[0] }), task('imports/parse-csv-chunk', { csv: chunks[1] }), task('imports/parse-csv-chunk', { csv: chunks[2] }), task('imports/parse-csv-chunk', { csv: chunks[3] }), ]) return [...a, ...b, ...c, ...d]}What you’ll see: on a machine with four or more cores, about a quarter of the time it takes on one thread, and the app keeps answering other requests meanwhile.
What can cross to another thread
Section titled “What can cross to another thread”A task runs in a separate JavaScript world. Its arguments and its result are copied, with the same rules as Node’s worker threads (the “structured clone” rules):
| Crosses fine | Doesn’t cross |
|---|---|
strings, numbers, booleans, null, undefined |
functions |
| plain objects and arrays | class instances keep their data but lose their methods (a Post comes back as a plain object) |
Date, Map, Set, ArrayBuffer, typed arrays |
open database connections, sockets, file handles |
So: give tasks plain data and return plain data. A task module imports whatever helpers it needs itself.
The team and the request travel with the task
Section titled “The team and the request travel with the task”When you call Concurrency.run() during a request, the current team (tenant) and the request context (request id, the user) are passed explicitly with each task, as data, and re-established in the worker before your task runs. Inside the task, the tenant scope protects queries exactly as in the request, and log lines carry the same request id.
import { Log } from '@strutvane/framework'import { Task } from '../../models/task.ts'
/** Summarises one project's tasks. The team was re-established before this runs. */export default async function build(args: { projectId: string }) { const tasks = await Task.query().where('project_id', args.projectId).get() // only this team's rows Log.info('Report built', { projectId: args.projectId, tasks: tasks.length }) return { projectId: args.projectId, open: tasks.filter((t) => t.status === 'open').length }}import { Concurrency, task } from '@strutvane/framework'
export function buildReports(projectIds: readonly string[]) { return Concurrency.run(projectIds.map((projectId) => task('reports/build', { projectId })))}What you’ll see: one result per project, each computed in a worker, each reading only the current team’s tasks. Each worker opens its own database connection, so keep the number of tasks per call modest (see Don’t start thousands at once). For pure number-crunching, loading the rows first on the main thread and passing them in is still the cheapest.
Workers or processes
Section titled “Workers or processes”| Driver | Runs tasks on | Takes | Good for |
|---|---|---|---|
| threads (worker threads) | other threads in the same process; cheap to start | module tasks only | most CPU-heavy work |
| processes | separate Node processes | module tasks only | work that may crash or use a lot of memory, so it can’t take the app down |
sync |
the main thread, one after the other | module tasks and closures | tests: deterministic and easy to debug |
A closure passed to a worker driver is refused with an error that tells you to move it into a task module.
When a task throws
Section titled “When a task throws”If a task throws, Concurrency.run() rejects with that error, like Promise.all():
try { await previewImport(chunks)} catch (error) { // e.g. "Row 1,204: expected 2 columns" thrown inside parse-csv-chunk}What you’ll see: the original error message from inside the task. If a thread or process dies outright (out of memory, a crash), you get an error that says so, something like:
ConcurrencyError: A task's worker exited unexpectedly (code 134).and the main app keeps running.
In tests: the sync driver
Section titled “In tests: the sync driver”In tests, run tasks on the main thread so they’re deterministic and your debugger can step into them. With sync you may also pass plain closures, which is handy for small checks:
import { describe, expect, it } from 'vitest'import { Concurrency } from '@strutvane/framework'import { previewImport } from '../../app/services/import-preview.ts'
describe('previewImport', () => { it('parses every chunk', async () => { Concurrency.driver('sync') const rows = await previewImport(['a,draft', 'b,published', 'c,draft', 'd,draft']) expect(rows).toHaveLength(4) })
it('accepts closures with the sync driver only', async () => { Concurrency.driver('sync') expect(await Concurrency.run([() => 1 + 1, () => 2 + 2])).toEqual([2, 4]) })})After the response: defer()
Section titled “After the response: defer()”Some work doesn’t need to happen before the user sees the page, but is too small for a queue: recording that a post was viewed, warming a cache entry. defer(fn) runs it right after the response has been sent:
import { defer, type HttpContext } from '@strutvane/framework'import { recordView } from '../../services/views.ts'
async show(ctx: HttpContext): Promise<Response> { const post = await this.visible(ctx) defer(() => recordView(post.id)) // runs after the response is on its way return ctx.json({ data: post })}What you’ll see: the response arrives without waiting for recordView. If recordView throws, the error is logged (with the request id), never lost, and the user, who already has their page, is unaffected.
defer() is not a queue: there are no retries, and if the server stops right after the response, the deferred work may not run. Use a queued job for anything that must happen.
With Docker or without Docker
Section titled “With Docker or without Docker”Everything on this page runs inside your app’s own process (and, for Concurrency.run(), its threads or child processes), so it works the same with and without Docker. Two things to know:
- Cores: threads only help up to the number of CPU cores the app can use. Docker Desktop on Windows and macOS gives containers a share of your cores (set in Docker Desktop’s resources settings); a production container may be limited too.
- Memory: each worker thread or process has its own memory. Four tasks that each load a 200 MB file need about 800 MB.
Common mistakes
Section titled “Common mistakes”Promise.all() made my page slower, not faster
Section titled “Promise.all() made my page slower, not faster”Why: the calls were CPU-heavy, not waiting. On one thread they still run one after the other.
Fix: for computation, use Concurrency.run() (0.3) or a queued job. Keep Promise.all() for database and HTTP calls.
The database refuses connections under load
Section titled “The database refuses connections under load”Error: Too many connectionsWhy: a Promise.all() over a long list started far more queries than the database allows at once.
Fix: work in chunks, or queue one job per item:
for (let start = 0; start < ids.length; start += 10) { await Promise.all(ids.slice(start, start + 10).map(recalculateProject))}My object came back from a task without its methods
Section titled “My object came back from a task without its methods”Why: data crossing to another thread is copied as plain data; class instances lose their methods. Fix: return plain data from tasks, and turn it back into what you need on the main thread.
“Closures can’t be sent to a worker thread”
Section titled ““Closures can’t be sent to a worker thread””Something like this:
ConcurrencyError: Closures can't be sent to a worker thread. Move the work into a task module and call task('imports/parse-csv-chunk', args).Why: the threads and processes drivers take module tasks only; a closure and the variables it uses can’t be copied to another thread. (Closures work only with the sync driver.)
Fix: move the code into a task module and pass plain arguments:
await Concurrency.run([task('imports/parse-csv-chunk', { csv })])A task can’t see a variable from my file
Section titled “A task can’t see a variable from my file”Why: a task module runs in a separate JavaScript world; it doesn’t share the variables of the code that started it. Fix: pass everything the task needs as plain arguments, and import helpers inside the task’s own module.
Work I deferred sometimes never happens
Section titled “Work I deferred sometimes never happens”Why: defer() runs after the response in the same process, with no retries; a restart or crash right after the response skips it.
Fix: if it must happen, dispatch a queued job instead.
Next steps
Section titled “Next steps”- Queues: reliable background work with retries.
- Processes: run other programs safely.
- Cache: avoid recomputing in the first place.
- ORM: Transactions: why writes in a transaction run one at a time.