Skip to content
GitHub

Strutvane 0.2 is still being built. These pages describe how it will work; anything marked "Coming in 0.2" doesn't exist yet. Release notes and support policy.

Concurrency

View as Markdownllms.txt

Concurrency means doing several things in overlapping time instead of one after the other. A dashboard that needs five numbers can ask for all five at once and wait about as long as the slowest one, instead of the sum of all five.

In a Strutvane app there are three tools for this, and picking the right one matters more than anything else on this page:

You want to… Use Built?
wait for several database queries or HTTP calls at once Promise.all() (plain JavaScript) yes, it’s part of the language
run CPU-heavy work (parsing a big file, crunching numbers) without freezing the app Concurrency.run([task(…), …]) on worker threads or processes planned for 0.3
do something after the response has been sent defer(fn) planned for 0.2
do something later, reliably, with retries a queued job planned for 0.3
run another program Process planned for 0.3

A stats endpoint needs three counts. Ask for all three at once:

app/http/controllers/stats.controller.ts
import type { HttpContext } from '@strutvane/framework'
import { Post } from '../../models/post.ts'
import { Tag } from '../../models/tag.ts'
import { User } from '../../models/user.ts'
export class StatsController {
async show(ctx: HttpContext): Promise<Response> {
const [posts, users, tags] = await Promise.all([
Post.query().count(),
User.query().count(),
Tag.query().count(),
])
return ctx.json({ posts, users, tags })
}
}

What you’ll see:

{ "posts": 128, "users": 14, "tags": 9 }

If each count takes 30 ms, the response takes about 30 ms instead of 90 ms. Promise.all() starts all three queries, then waits until every one has answered, and gives you the answers in the same order you listed them.

Node runs your code on one thread (one line of execution). That sounds limiting, but most of a web request is waiting: for the database, for another API, for a file. While one query waits, Node is free to start the next one. That’s why Promise.all() gives real speed-ups for database and HTTP work without any threads.

The work this does not help with is CPU-bound work: code that keeps the processor busy (resizing a big image in JavaScript, parsing a 200 MB CSV, hashing thousands of values). While it runs, that thread can do nothing else, so every other request on the server waits. That’s what Concurrency.run() is for.

If one call fails, Promise.all() fails at once with that error. Use it when you need every answer:

app/services/project-summary.ts
import { Project } from '../models/project.ts'
import { Task } from '../models/task.ts'
export async function projectSummary(projectId: string) {
const [project, openTasks] = await Promise.all([
Project.findOrFail(projectId), // throws if missing
Task.query().where('project_id', projectId).where('status', 'open').count(),
])
return { name: project.name, openTasks }
}

What you’ll see for an unknown project: the ModelNotFoundError from findOrFail, which your app turns into a 404, exactly as if you’d awaited the calls one by one. The other query still runs to the end in the background; its answer is thrown away.

When each part is optional (a dashboard where one widget may fail), use Promise.allSettled(). It never fails; it tells you which parts worked:

app/services/dashboard.ts
import { Log } from '@strutvane/framework'
import { Post } from '../models/post.ts'
import { fetchPartnerStatus } from './partner.ts'
export async function dashboard() {
const [posts, partner] = await Promise.allSettled([
Post.query().where('status', 'published').count(),
fetchPartnerStatus(), // another company's API: may be down
])
if (partner.status === 'rejected') Log.warning('Partner status unavailable', { error: partner.reason })
return {
published: posts.status === 'fulfilled' ? posts.value : null,
partner: partner.status === 'fulfilled' ? partner.value : 'unavailable',
}
}

What you’ll see when the partner API is down:

{ "published": 128, "partner": "unavailable" }

The page still loads; only the partner widget shows “unavailable”.

Promise.all() over a list of 10,000 items starts 10,000 queries at the same moment. Your database has a limited number of connections, so most of them queue up, and the database may refuse new ones. Work in chunks:

app/services/recalculate.ts
import { recalculateProject } from './recalculate-project.ts'
/** At most 10 projects at a time. */
export async function recalculateAll(projectIds: readonly string[]): Promise<void> {
for (let start = 0; start < projectIds.length; start += 10) {
const chunk = projectIds.slice(start, start + 10)
await Promise.all(chunk.map((id) => recalculateProject(id)))
}
}

For really large amounts of work, push one queued job per item (or a batch) and let workers spread it out.

A transaction runs on one database connection, and a connection runs one statement at a time. Inside transaction(), await each write in turn instead of using Promise.all(): the writes can’t overlap anyway, and keeping them in order makes failures easy to follow.

Concurrency.run() takes a list of tasks, runs each one on another thread or process, and returns their results in order. The main thread stays free to answer other requests while they work.

A task for a worker is a module, not a closure. A closure (a function that uses variables from the code around it) can’t be copied to another thread, because the other thread has its own memory. So each task lives in its own file and is named by its path; you hand it plain arguments.

Step 1: write the task module.

app/tasks/imports/parse-csv-chunk.ts
/** Parses one chunk of an uploaded CSV. Runs on a worker: plain data in, plain data out. */
export default async function parseCsvChunk(args: { csv: string }): Promise<Array<{ title: string; status: string }>> {
return args.csv
.split('\n')
.filter((line) => line.trim() !== '')
.map((line, index) => {
const [title, status] = line.split(',')
if (title === undefined || status === undefined) throw new Error(`Row ${index + 1}: expected 2 columns`)
return { title, status }
})
}

Step 2: run it four times at once.

app/services/import-preview.ts
import { Concurrency, task } from '@strutvane/framework'
/** Parses a large upload in four parts at once, on other threads. */
export async function previewImport(chunks: readonly [string, string, string, string]) {
const [a, b, c, d] = await Concurrency.run([
task('imports/parse-csv-chunk', { csv: chunks[0] }),
task('imports/parse-csv-chunk', { csv: chunks[1] }),
task('imports/parse-csv-chunk', { csv: chunks[2] }),
task('imports/parse-csv-chunk', { csv: chunks[3] }),
])
return [...a, ...b, ...c, ...d]
}

What you’ll see: on a machine with four or more cores, about a quarter of the time it takes on one thread, and the app keeps answering other requests meanwhile.

A task runs in a separate JavaScript world. Its arguments and its result are copied, with the same rules as Node’s worker threads (the “structured clone” rules):

Crosses fine Doesn’t cross
strings, numbers, booleans, null, undefined functions
plain objects and arrays class instances keep their data but lose their methods (a Post comes back as a plain object)
Date, Map, Set, ArrayBuffer, typed arrays open database connections, sockets, file handles

So: give tasks plain data and return plain data. A task module imports whatever helpers it needs itself.

The team and the request travel with the task

Section titled “The team and the request travel with the task”

When you call Concurrency.run() during a request, the current team (tenant) and the request context (request id, the user) are passed explicitly with each task, as data, and re-established in the worker before your task runs. Inside the task, the tenant scope protects queries exactly as in the request, and log lines carry the same request id.

app/tasks/reports/build.ts
import { Log } from '@strutvane/framework'
import { Task } from '../../models/task.ts'
/** Summarises one project's tasks. The team was re-established before this runs. */
export default async function build(args: { projectId: string }) {
const tasks = await Task.query().where('project_id', args.projectId).get() // only this team's rows
Log.info('Report built', { projectId: args.projectId, tasks: tasks.length })
return { projectId: args.projectId, open: tasks.filter((t) => t.status === 'open').length }
}
app/services/reports.ts
import { Concurrency, task } from '@strutvane/framework'
export function buildReports(projectIds: readonly string[]) {
return Concurrency.run(projectIds.map((projectId) => task('reports/build', { projectId })))
}

What you’ll see: one result per project, each computed in a worker, each reading only the current team’s tasks. Each worker opens its own database connection, so keep the number of tasks per call modest (see Don’t start thousands at once). For pure number-crunching, loading the rows first on the main thread and passing them in is still the cheapest.

Driver Runs tasks on Takes Good for
threads (worker threads) other threads in the same process; cheap to start module tasks only most CPU-heavy work
processes separate Node processes module tasks only work that may crash or use a lot of memory, so it can’t take the app down
sync the main thread, one after the other module tasks and closures tests: deterministic and easy to debug

A closure passed to a worker driver is refused with an error that tells you to move it into a task module.

If a task throws, Concurrency.run() rejects with that error, like Promise.all():

app/services/import-preview.ts
try {
await previewImport(chunks)
} catch (error) {
// e.g. "Row 1,204: expected 2 columns" thrown inside parse-csv-chunk
}

What you’ll see: the original error message from inside the task. If a thread or process dies outright (out of memory, a crash), you get an error that says so, something like:

ConcurrencyError: A task's worker exited unexpectedly (code 134).

and the main app keeps running.

In tests, run tasks on the main thread so they’re deterministic and your debugger can step into them. With sync you may also pass plain closures, which is handy for small checks:

tests/unit/import-preview.test.ts
import { describe, expect, it } from 'vitest'
import { Concurrency } from '@strutvane/framework'
import { previewImport } from '../../app/services/import-preview.ts'
describe('previewImport', () => {
it('parses every chunk', async () => {
Concurrency.driver('sync')
const rows = await previewImport(['a,draft', 'b,published', 'c,draft', 'd,draft'])
expect(rows).toHaveLength(4)
})
it('accepts closures with the sync driver only', async () => {
Concurrency.driver('sync')
expect(await Concurrency.run([() => 1 + 1, () => 2 + 2])).toEqual([2, 4])
})
})

Some work doesn’t need to happen before the user sees the page, but is too small for a queue: recording that a post was viewed, warming a cache entry. defer(fn) runs it right after the response has been sent:

app/http/controllers/post.controller.ts (excerpt)
import { defer, type HttpContext } from '@strutvane/framework'
import { recordView } from '../../services/views.ts'
async show(ctx: HttpContext): Promise<Response> {
const post = await this.visible(ctx)
defer(() => recordView(post.id)) // runs after the response is on its way
return ctx.json({ data: post })
}

What you’ll see: the response arrives without waiting for recordView. If recordView throws, the error is logged (with the request id), never lost, and the user, who already has their page, is unaffected.

defer() is not a queue: there are no retries, and if the server stops right after the response, the deferred work may not run. Use a queued job for anything that must happen.

Everything on this page runs inside your app’s own process (and, for Concurrency.run(), its threads or child processes), so it works the same with and without Docker. Two things to know:

  • Cores: threads only help up to the number of CPU cores the app can use. Docker Desktop on Windows and macOS gives containers a share of your cores (set in Docker Desktop’s resources settings); a production container may be limited too.
  • Memory: each worker thread or process has its own memory. Four tasks that each load a 200 MB file need about 800 MB.

Promise.all() made my page slower, not faster

Section titled “Promise.all() made my page slower, not faster”

Why: the calls were CPU-heavy, not waiting. On one thread they still run one after the other. Fix: for computation, use Concurrency.run() (0.3) or a queued job. Keep Promise.all() for database and HTTP calls.

The database refuses connections under load

Section titled “The database refuses connections under load”
Error: Too many connections

Why: a Promise.all() over a long list started far more queries than the database allows at once. Fix: work in chunks, or queue one job per item:

for (let start = 0; start < ids.length; start += 10) {
await Promise.all(ids.slice(start, start + 10).map(recalculateProject))
}

My object came back from a task without its methods

Section titled “My object came back from a task without its methods”

Why: data crossing to another thread is copied as plain data; class instances lose their methods. Fix: return plain data from tasks, and turn it back into what you need on the main thread.

“Closures can’t be sent to a worker thread”

Section titled ““Closures can’t be sent to a worker thread””

Something like this:

ConcurrencyError: Closures can't be sent to a worker thread. Move the work into a task module and call task('imports/parse-csv-chunk', args).

Why: the threads and processes drivers take module tasks only; a closure and the variables it uses can’t be copied to another thread. (Closures work only with the sync driver.) Fix: move the code into a task module and pass plain arguments:

await Concurrency.run([task('imports/parse-csv-chunk', { csv })])

A task can’t see a variable from my file

Section titled “A task can’t see a variable from my file”

Why: a task module runs in a separate JavaScript world; it doesn’t share the variables of the code that started it. Fix: pass everything the task needs as plain arguments, and import helpers inside the task’s own module.

Why: defer() runs after the response in the same process, with no retries; a restart or crash right after the response skips it. Fix: if it must happen, dispatch a queued job instead.

  • Queues: reliable background work with retries.
  • Processes: run other programs safely.
  • Cache: avoid recomputing in the first place.
  • ORM: Transactions: why writes in a transaction run one at a time.