Initial commit

This commit is contained in:
robert committed 2025-05-30 18:13:30 +08:00
commit e49726988c
5445 files changed
+1089914

No files matched your search

+752
View File
@@ -0,0 +1,752 @@
# Changelog
## Versioning
See [Semver](http://semver.org/).
### The `MAJOR` or `API` version is incremented for
- 💔 Non-backwards-compatible API changes
### The `MINOR` or `UPDATE` version is incremented for
- ✨ Backwards-compatible features
### The `PATCH` version is incremented for
- 🐞 Backwards-compatible bug fixes
- 📦 Minor packaging changes
## v13.0.0
- 💔 Dropped official support for [Node v16, which is EOL](https://nodejs.org/en/blog/announcements/nodejs16-eol/).
- 💔 Several methods, including BatchCluster#pids() were changed from async to sync (as they were needlessly async).
- 📦 A number of timeout options can now be validly 0 to disable timeouts:
- `spawnTimeoutMillis`
- `taskTimeoutMillis`
- 📦 Added eslint `@typescript-eslint/await-thenable` rule and delinted.
- 📦 Updated development dependencies and rebuilt docs
## v12.1.0
- 🐞 `pidExists` now handles `EPERM` properly (previous implementation would
mischaracterize pids as being dead due to insufficient permissions)
- 📦 Updated development dependencies and rebuilt docs
## v12.0.0
- 💔/✨ `pidExists` and `killPid` are no longer `async`, as process management
is now performed via `node:process.kill()`, instead of forking `ps` or `tasklist`.
- 📦 Updated development dependencies and rebuilt docs
## v11.0.0
- 💔 Drop official support for Node 12: [EOL was 2022-04-30](https://github.com/nodejs/release#end-of-life-releases)
## v10.4.3
- 🐞 Fix support for zero value of `maxProcAgeMillis`
- 📦 Updated development dependencies and rebuilt docs
## v10.4.2
- 🐞 Fix [`unref` is not a function](https://github.com/photostructure/batch-cluster.js/issues/16)
- 📦 Updated development dependencies and rebuilt docs
## v10.4.1
- 📦 Improved concurrent event `Rate` measurement.
## v10.4.0
- ✨ If `healthCheckCommand` is set and any task fails, that child process will
have a health check run before being put back into rotation.
- 📦 Updated development dependencies and rebuilt docs
## v10.3.2
- 🐞 `BatchCluster#maybeSpawnProcs` in prior versions could spawn too many
processes, especially if process startup was slow. Heuristics for when to
spawn new processes now take into account pending task length and processes
busy due to initial setup.
- 📦 `BatchCluster.vacuumProcs` returns a promise that is only fulfilled after
all reaped child processes have completed `BatchProcess.#end`.
- 📦 `BatchProcess.whyNotHealthy` can now return `startError`.
- 📦 `childEnd` is now emitted only after the child process exits
- 📦 `BatchCluster.#onIdle` is debounced during the same event loop
- 📦 Added startup and shutdown spec assertions
- 📦 Updated development dependencies and rebuilt docs
## v10.3.1
- 📦 Add `Rate.msSinceLastEvent`
- 📦 Adjusted `streamFlushMillis` to remove `onTaskData` errors in CI.
## v10.3.0
- ✨ Exported `Rate`. You might like it.
- ✨ When child processes emit to `stdout` or `stderr` with no current task,
prior versions would emit an `internalError`. These are now given their own
new `noTaskData` events. Consumers may want to bump up `streamFlushMillis` if
they see this in production.
- 🐞/📦 Increased defaults for `streamFlushMillis`, added tests to verify `noTaskData` events don't happen in CI.
- 📦 Normalized node imports
## v10.2.0
- ✨/📦 Set `minDelayBetweenSpawnMillis = 0` to fork child processes as soon as
they are needed (rather than waiting between `spawn` calls)
- ✨/📦 Set `maxReasonableProcessFailuresPerMinute = 0` to disable process start
error rate detection.
- ✨/📦 New `fatalError` event emitted when
`maxReasonableProcessFailuresPerMinute` is exceeded and the instance shuts
itself down.
- 📦 New simpler `Rate` implementation with better time decay handling
- 📦 Several jsdoc improvements, including exporting `WhyNotHeathy` and
`WhyNotReady`
## v10.1.1
- 🐞 Fixed [issue
#15](https://github.com/photostructure/batch-cluster.js/issues/15) by
restoring the call to `#onIdleLater` when tasks settle.
- 🐞 Fixed issue with `setMaxProcs` which resulted in all idle processes being
reaped
- 📦 The `idle` event was removed. _You weren't using it, though, so I'm not
bumping major._
- 📦 Process shutdown is handled more gracefully with new `thenOrTimeout`
(rather than the prior `Promise.race` call which resulted in a dangling
timeout)
- 📦 Updated development dependencies and rebuilt docs
## v10.1.0
- 📦 `.end()` and `.closeChildProcesses()` closes all child processes in parallel
## v10.0.1
- 📦 Export `BatchProcess` interface
## v10.0.0
- ✨ Process state improvements
- 💔 Renamed event s/childExit/childEnd/
- 💔 `childEnd` and `childStart` events receive BatchProcess instances now
- 💔 Renamed healthy state s/dead/ended/
- 📦 Made BatchProcess.whyNotHealthy persistent
- 📦 Added several more WhyNotHealthy values
- 📦 Perf: filterInPlace and count use for loops rather than closures
- 📦 Added spec to verify `.end` rejects long-running pending tasks
- 📦 Updated development dependencies and rebuilt docs
## v9.1.0
- 🐞/📦 `BatchProcess` exposes a promise for the completion of the startup task,
which `BatchCluster` now uses to immediately run `#onIdle` and pop off any
pending work.
- 📦 Updated development dependencies and rebuild docs
## v9.0.1
- 📦 Don't emit `taskResolved` on startup tasks.
## v9.0.0
- 💔 The `BatchProcessObserver` signature was deleted, as `BatchClusterEmitter` is
now typesafe. Consumers should not have used this signature directly, but in
case anyone did, I bumped the major version.
- ✨ Added `BatchCluster.off` to unregister event listeners provided to `BatchCluster.on`.
- 📦 Private fields and methods now use [the `#` private
prefix](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_class_fields)
rather than the TypeScript `private` modifier.
- 📦 Minor tweaks (fixed several jsdoc errors, simplified some boolean logic,
small reduction in promise chains, ...)
- 📦 Updated development dependencies and rebuild docs
## v8.1.0
- 📦 Added `BatchCluster.procCount` and `BatchCluster.setMaxProcs`, and new
`BatchCluster.ChildEndCountType` which includes a new `tooMany` value, which
is incremented when `setMaxProcs` is set to a smaller value.
- 📦 Updated development dependencies
## v8.0.1
- 🐞/📦 BatchProcess now end on spurious stderr/stdout, and reject tasks if ending.
- 📦 Relaxed default for `streamFlushMillis` to deflake CI
## v8.0.0
- 💔/📦 RegExp pass/fail strings are escaped (which could conceivably be a breaking change, hence the major version bump)
- 📦 Refactored stdout/stderr merging code and added more tests
- 📦 Added new "taskResolved" event
- 📦 Rebuild docs
- 📦 Updated development dependencies
## v7.2.1
- 📦 Relax typing for optional `BatchProcessOptions` fields
## v7.2.0
- 📦 Upgrade all dev dependencies. Pulling in new TypeScript 4.4 required [redoing all node imports](https://github.com/microsoft/TypeScript/issues/46027#issuecomment-926019016).
## v7.1.0
- ✨ Added `on("healthCheckError", err, proc)` event
- 🐞 Fixed process start lag (due to startup tasks not emitting an `.onIdle`)
- 🐞 Reworked when health checks were run, and add tests to validate failing health checks recycle children
- 📦 Rebuild docs
## v7.0.0
- 💔 Several fields were renamed to make things more consistent:
- `BatchCluster.pendingTasks` was renamed to `BatchCluster.pendingTaskCount`.
- A new `BatchCluster.pendingTasks` method now matches `BatchCluster.currentTasks`, which both return `Task[]`.
- `BatchCluster.busyProcs` was renamed to `busyProcCount`.
- `BatchCluster.spawnedProcs` was renamed to `spawnedProcCount`.
- ✨ Added support for "health checks" that run periodically on child processes.
Both `healthCheckCommand` and `healthCheckIntervalMillis` must be set to
enable this feature.
- ✨ New `pidCheckIntervalMillis` to verify internal child process state is kept
in sync with the process table. Defaults to every 2 minutes. Will no-op if idle.
- ✨ New `BatchCluster.childEndCounts` to report why child processes were recycled (currently "dead" | "ending" | "closed" | "worn" | "idle" | "broken" | "old" | "timeout" )
- 📦 Cleaned up scheduling: the prior implementation generated a bunch of
`Promise`s per idle period, which was rough on the GC. Use of `Mutex` is now
relegated to tests.
- 📦 `tsconfig` now emits `ES2018` output and doesn't have `downlevelIteration`,
which reduces the size of the generated javascript, but requires contemporary
versions of Node.js.
- 📦 `BatchClusterOptions` doesn't mark fields as `readonly` anymore
- 📦 `Task` has a default type of `any` now.
## v6.2.1
- 📦 Added `BatchCluster.currentTasks`
## v6.2.0
- 📦 Updated development dependencies, which required handling undefined process ids.
## v6.1.0
- ✨ Added `BatchCluster.closeChildProcesses()` (ends child processes but doesn't `.end()` the BatchCluster instance)
- 📦 Updated development dependencies
## v6.0.2
- 📦 Include sourcemaps
- 📦 Updated development dependencies
## v6.0.1
- 📦 Updated development dependencies
- 📦 Renamed `main` branch
- 📦 Hopefully fixed all typedoc URL changes
## v6.0.0
No new features in v6: just a breaking change so we can fix an old name
collision that caused linting errors.
- 💔 Prior versions name-collided on `Logger`: both as an `interface` and as a
pseudonamespace for logger factories. This made `eslint` grumpy, and if anyone
actually used this bare-bones logger, it could have caused confusion.
`Logger` now references _only the `interface`._
The builder functions are now named `Log`.
- 📦 Updated development dependencies
## v5.11.3
- 📦 Updated development dependencies (primarily TypeScript 4.1)
- 📦 `Deferred.resolve` now requires an argument (as per the new Promise spec).
As this is just a typing change (and `Deferred` is an internal
implementation), I'm not bumping the major version.
## v5.11.2
- 📦 Updated development dependencies
- 📦 Minor delint/prettier reformat
## v5.11.1
- 📦 Updated development dependencies
## v5.11.0
- ✨ `BatchCluster` can now be created with a `Logger` thunk.
- 📦 De-linted
- 📦 Updated development dependencies
- 📦 Add Node v14 to build matrix
## v5.10.0
- ✨ New `maxIdleMsPerProcess` option: automatically shut down idle child
processes to reduce system resource consumption. Defaults to `0`, which
disables this feature (and prevents me from having to increment the major
version!)
- 📦 Updated development dependencies
## v5.9.5
- 📦 Updated development dependencies
- 📦 Ran prettier (2.0.0 causes many no-op diffs due to changed defaults)
## v5.9.4
- 📦 Updated development dependencies
## v5.9.3
- 🐞 `BatchProcess`'s streams could cause an infinite loop on `.end()` when
`stdout` was destroyed.
- 📦 Updated development dependencies
## v5.9.2
- 🐞 `BatchProcess.ready` now verifies the child process still exists
- 📦 Replace tslint with eslint
- 📦 Updated development dependencies
## v5.9.1
- 🐞 Errors after a process has shut down are logged and not propagated
- 📦 Updated development dependencies
## v5.9.0
- 🐞 Moved all async throws into observables (to prevent "This error originated
either by throwing inside of an async function without a catch block, or by
rejecting a promise which was not handled with .catch(). The promise rejected
with the reason...")
- 📦 Updated development dependencies
## v5.8.0
- 🐞 Fixed issue where immediately closing a process before a pending task
completed resulted in `Error: onExit(exit) called end()`
- 📦 Updated development dependencies
## v5.7.1
- 📦 `BatchCluster.end()` should return a `Deferred<void>`
## v5.7.0
- 🐞 Fixed issue where `onStartError` and `onTaskError` didn't get emitted.
- 📦 Updated development dependencies, rebuilt docs.
- 🐞 Deflaked CI tests with longer timeouts and less aggressive `shutdown()`
- 📦 Had to delete the macOS Travis tests. Travis has been terribly flaky, with
unreproduceable spec failures.
## v5.6.8
- 📦 Updated development dependencies (new TypeScript)
## v5.6.7
- 📦 Updated development dependencies
## v5.6.6
- 📦 Updated development dependencies
## v5.6.5
- 📦 wrapped `stdin.write()` with try/catch that rejects the current task and
closes the current process.
- 📦 wrapped `stdin.end()` with try/catch (as `.writable` isn't reliable)
## v5.6.4
- 📦 Updated development dependencies
## v5.6.3
- 📦 Moved to the PhotoStructure org. Updated URLs in docs.
## v5.6.2
- 📦 Updated development dependencies
- 📦 Removed trace and debug log calls in `BatchProcess` (which incurred GC
overhead even when disabled)
## v5.6.1
- 📦 Expose `BatchCluster.options`. Note that the object is frozen at
construction.
## v5.6.0
- 🐞/✨ `BatchProcess.end()` didn't correctly implement `gracefully` (which
resulted in spurious `end(): called while not idle` errors), and allowed for
multiple calls to destroy and disconnect from the child process, which may or
may not have been ill-advised.
## v5.5.0
- ✨ Added `BatchCluster.isIdle`. Updated development dependencies. Deflaked CI by embiggening
- ✨ Added `BatchClusterOptions.cleanupChildProcs`, in case you want to handle
process cleanup yourself.
- 📦 Updated development dependencies. Deflaked CI by embiggening timeouts.
- Happy 🥧 day.
## v5.4.0
- ✨ "wear-leveling" for processes. Previously, only the first-spawned child
process would service most task requests, but that caused issues with (very)
long-running tasks where the other child processes would be spooled off ram,
and could time out when requested later.
- 🐞 `maxProcs` is respected again. In prior builds, if tasks were enqueued all
at once, prior dispatch code would only spin 1 concurrent task at a time.
- 🐞 Multiple calls to `BatchProcess.end` would result in different promise
resolution targets: the second call to `.end()` would resolve before the
first. This was fixed.
- ✨
[BatchProcessOptions](https://batch-cluster.js.org/classes/batchclusteroptions.html)'s
`minDelayBetweenSpawnMillis` was added, to help relieve undue system load on
startup. It defaults to 1.5 seconds and can be disabled by setting it to 0.
## v5.3.1
- 📦 Removed `Deferred`'s warn log messages.
## v5.3.0
- 🐞 `.pass` and `.fail` regex now support multiple line outputs per task.
## v5.2.0
- 🐞 [BatchProcessOptions](https://batch-cluster.js.org/classes/batchclusteroptions.html)`.pass`
and `.fail` had poorly specified and implemented failure semantics. Prior
implementations would capture a "failed" string, but not tell the task that
the service returned a failure status.
Task [Parser](https://batch-cluster.js.org/interfaces/parser.html)s already
accept stdout and stderr, and are the "final word" in resolving or rejecting
[Task](https://batch-cluster.js.org/classes/task.html)s.
`v5.2.0` provides a boolean to Parser's callable indicating if the wrapped
service returned pass or fail, and the Parser may return a Promise now, as
well.
There's a new `SimpleParser` implementation you can use that fails if `stderr`
is non-blank or a stream matched the `.fail` pattern.
- 🐞 initial `BatchProcess` validation uses the new `SimpleParser` to verify the
initial `versionCommand`.
- ✨ child process pids are delivered to event listeners on spawn and close. See
[BatchClusterEmitter](https://batch-cluster.js.org/classes/batchclusteremitter.html).
- 🐞 fix "Error: end() called when not idle" by debouncing stdout and stderr
readers. Note that this adds latency to every task. See
[BatchProcessOptions](https://batch-cluster.js.org/classes/batchclusteroptions.html)'s
`streamFlushMillis` option, which defaults to 10 milliseconds.
- 🐞 RegExp for pass and fail tokens handle newline edge cases now.
- 📦 re-added tslint and delinted code.
## v5.1.0
- ✨ `ChildProcessFactory` supports thunks that return either a `ChildProcess` or
`Promise<ChildProcess>`
- 📦 Update deps
## v5.0.1
- 📦 Update deps
- 📦 re-run prettier
## v5.0.0
- 💔 The `rejectTaskOnStderr` API, which was added in v4.1.0 and applied to all
tasks for a given `BatchCluster` instance, proved to be a poor decision, and
has been removed. The `Parser` API, which is task-specific, now receives
**both** stdin and stderr streams. Parsers then have the necessary context to
decide what to do on a per task or per task-type basis.
- 🐞 In previous versions, batch processes were recycled if any task had any
type of error. This version allows pids to live even if they emit data to
stderr.
## v4.3.0
- ✨ If your tasks return interim progress and you want to capture that data
as it happens, BatchCluster now emits `taskData` events with the data and the
current task (which may be undefined) as soon as the stream data is emitted.
- 📦 Pulled in latest dependency versions
## v4.2.0
- 📦 In the interests of less noise, the default logger is now the `NoLogger`.
Consumers may use the `ConsoleLogger` or another `Logger` implementation as
they see fit.
## v4.1.0
- ✨ Support for demoting task errors from `stderr` emissions:
`BatchProcess.rejectTaskOnStderr` is a per-task, per-error predicate which
allows for a given error to be handled without always rejecting the task. This
can be handy if the script you're wrapping (like ExifTool) writes non-fatal
warnings to stderr.
- ✨ `BatchProcessOptions.pass` and `BatchProcessOptions.fail` can be RegExp
instances now, if you have more exotic parsing needs.
## v4.0.0
- 💔 Using Node 8+ to determine if a process is running with `kill(pid, 0)`
turns out to be unreliable (as it returns true even after the process exits).
I tried to pull in the best-maintained "process-exists" external dependency,
but that pulled in 15 more modules (this used to be a zero-deps module), and
it was extremely unperformant on Windows.
The TL;DR: is that `running(pid)` now returns a `Promise<boolean>`, which had
far-reaching signature changes to accomodate the new asynchronicity, hence the
major version bump.
- 💔 In an effort to reduce this library's complexity, I'm removing retry
functionality. All parameters associated to retries are now gone.
- ✨ Internal state validation is now exposed by BatchCluster, and is used by
tests to ensure no internal errors happen during integration tests. Previously
these errors were simply logged.
## v3.2.0
- 📦 New `Logger` methods, `withLevels`, `withTimestamps`, and `filterLevels`
were shoved into a new `Logger` namespace.
## v3.1.0
- ✨ Added simple timestamp and levels logger prefixer for tests
- 🐞 Errors rethrown via BatchProcess now strip extraneous `Error:` prefixes
- 🐞 For a couple internal errors (versionCommend startup errors and internal
state inconsistencies on `onExit` that aren't fatal), we now log `.error`
rather than throw Error() or ignore.
## v3.0.0
- ✨/💔 **`Task` promises are only rejected with `Error` instances now.** Note
that also means that `BatchProcessObserver` types are more strict. It could be
argued that this isn't an API breaking change as it only makes rejection
values more strict, but people may need to change their error handling, so I'm
bumping the major version to highlight that. Resolves
[#3](https://github.com/mceachen/batch-cluster.js/issues/3). Thanks for the
issue, [Nils Knappmeier](https://github.com/nknapp)!
## v2.2.0
- 🐞 Windows taskkill `/PID` option seemed to work downcased, but the docs say
to use uppercase, so I've updated it.
- 📦 Upgrade all deps including TypeScript to 2.9
(v2.1.2 is the same contents, but `np` had a crashbug during publish)
## v2.1.1
- 📦 More robust `end` for `BatchProcess`, which may prevent very long-lived
consumers from sporadically leaking child processes on Mac and linux.
- 📦 Added Node 10 to the build matrix.
## v2.1.0
- 📦 Introduced `Logger.trace` and moved logging related to per-task items down
to `trace`, as heavy load and large request or response payloads could
overwhelm loggers. If you really want to see on-the-wire requests and results,
enable `trace` in your debugger implementation. By default, the
`ConsoleLogger` omits log messages with this level.
## v2.0.0
- 💔 Replaced `BatchClusterObserver` with a simple EventEmitter API on
`BatchCluster` to be more idiomatic with node's API
- 💔 v1.11.0 added "process reuse" after errors, but that turned out to be
problematic in recovery, so that change was reverted (and with it, the
`maxTaskErrorsPerProcess` parameter was removed)
- ✨ `Rate` is simpler and more accurate now.
## v1.11.0
- ✨ Added new `BatchClusterObserver` for error and lifecycle monitoring
- 📦 Added a number of additional logging calls
## v1.10.0
- 🐞 Explicitly use `timers.setInterval`. May address [this
issue](https://stackoverflow.com/questions/48961238/electron-setinterval-implementation-difference-between-chrome-and-node).
Thanks for the PR, [Tim Fish](https://github.com/timfish)!
## v1.9.1
- 📦 Changed `BatchProcess.end()` to use `until()` rather than `Promise.race`,
and always use `kill(pid, forced)` after waiting the shutdown grace period
to prevent child process leaks.
## v1.9.0
- ✨ New `Logger.setLogger()` for debug, info, warning, and errors. `debug` and
`info` defaults to Node's
[debuglog](https://nodejs.org/api/util.html#util_util_debuglog_section),
`warn` and `error` default to `console.warn` and `console.error`,
respectively.
- 📦 docs generated by [typedoc](http://typedoc.org/)
- 📦 Upgraded dependencies (including TypeScript 2.7, which has more strict
verifications)
- 📦 Removed tslint, as `tsc` provides good lint coverage now
- 📦 The code is now [prettier](https://github.com/prettier/prettier)
- 🐞 `delay` now allows
[unref](https://nodejs.org/api/timers.html#timers_timeout_unref)ing the
timer, which, in certain circumstances, could prevent node processes from
exiting gracefully until their timeouts expired
## v1.8.0
- ✨ onIdle now runs as many tasks as it can, rather than just one. This should
provide higher throughput.
- 🐞 Removed stderr emit on race condition between onIdle and execTask. The
error condition was already handled appropriately--no need to console.error.
## v1.7.0
- 📦 Exported `kill()` and `running()` from `BatchProcess`
## v1.6.1
- 📦 De-flaked some tests on mac, and added Node 8 to the build matrix.
## v1.6.0
- ✨ Processes are forcefully shut down with `taskkill` on windows and `kill -9`
on other unix-like platforms if they don't terminate after sending the
`exitCommand`, closing `stdin`, and sending the proc a `SIGTERM`. Added a test
harness to exercise.
- 📦 Upgrade to TypeScript 2.6.1
- 🐞 `mocha` tests don't require the `--exit` hack anymore 🎉
## v1.5.0
- ✨ `.running()` works correctly for PIDs with different owners now.
- 📦 `yarn upgrade --latest`
## v1.4.2
- 📦 Ran code through `prettier` and delinted
- 📦 Massaged test assertions to pass through slower CI systems
## v1.4.1
- 📦 Replaced an errant `console.log` with a call to `log`.
## v1.4.0
- 🐞 Discovered `maxProcs` wasn't always utilized by `onIdle`, which meant in
certain circumstances, only 1 child process would be servicing pending
requests. Added breaking tests and fixed impl.
## v1.3.0
- 📦 Added tests to verify that the `kill(0)` calls to verify the child
processes are still running work across different node version and OSes
- 📦 Removed unused methods in `BatchProcess` (whose API should not be accessed
directly by consumers, so the major version remains at 1)
- 📦 Switched to yarn and upgraded dependencies
## v1.2.0
- ✨ Added a configurable cleanup signal to ensure child processes shut down on
`.end()`
- 📦 Moved child process management from `BatchCluster` to `BatchProcess`
- ✨ More test coverage around batch process concurrency, reuse, flaky task
retries, and proper process shutdown
## v1.1.0
- ✨ `BatchCluster` now has a force-shutdown `exit` handler to accompany the
graceful-shutdown `beforeExit` handler. For reference, from the
[Node docs](https://nodejs.org/api/process.html#process_event_beforeexit):
> The 'beforeExit' event is not emitted for conditions causing explicit
> termination, such as calling process.exit() or uncaught exceptions.
- ✨ Remove `Rate`'s time decay in the interests of simplicity
## v1.0.0
- ✨ Integration tests now throw deterministically random errors to simulate
flaky child procs, and ensure retries and disaster recovery work as expected.
- ✨ If the `processFactory` or `versionCommand` fails more often than a given
rate, `BatchCluster` will shut down and raise exceptions to subsequent
`enqueueTask` callers, rather than try forever to spin up processes that are
most likely misconfigured.
- ✨ Given the proliferation of construction options, those options are now
sanity-checked at construction time, and an error will be raised whose message
contains all incorrect option values.
## v0.0.2
- ✨ Added support and explicit tests for
[CR LF, CR, and LF](https://en.wikipedia.org/wiki/Newline) encoded streams
from spawned processes
- ✨ child processes are ended after `maxProcAgeMillis`, and restarted as needed
- 🐞 `BatchCluster` now practices good listener hygene for `process.beforeExit`
## v0.0.1
- ✨ Extracted implementation and tests from
[exiftool-vendored](https://github.com/mceachen/exiftool-vendored.js)
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2017-2024 Matthew McEachen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+94
View File
@@ -0,0 +1,94 @@
# batch-cluster
**Efficient, concurrent work via batch-mode command-line tools from within Node.js.**
[![npm version](https://img.shields.io/npm/v/batch-cluster.svg)](https://www.npmjs.com/package/batch-cluster)
[![Build status](https://github.com/photostructure/batch-cluster.js/actions/workflows/node.js.yml/badge.svg?branch=main)](https://github.com/photostructure/batch-cluster.js/actions/workflows/node.js.yml)
[![GitHub issues](https://img.shields.io/github/issues/photostructure/batch-cluster.js.svg)](https://github.com/photostructure/batch-cluster.js/issues)
[![CodeQL](https://github.com/photostructure/batch-cluster.js/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/photostructure/batch-cluster.js/actions/workflows/codeql-analysis.yml)
[![Known Vulnerabilities](https://snyk.io/test/github/photostructure/batch-cluster.js/badge.svg?targetFile=package.json)](https://snyk.io/test/github/photostructure/batch-cluster.js?targetFile=package.json)
Many command line tools, like
[ExifTool](https://sno.phy.queensu.ca/~phil/exiftool/),
[PowerShell](https://github.com/powershell/powershell), and
[GraphicsMagick](http://www.graphicsmagick.org/), support running in a "batch
mode" that accept a series of discrete commands provided through stdin and
results through stdout. As these tools can be fairly large, spinning them up can
be expensive (especially on Windows).
This module allows you to run a series of commands, or `Task`s, processed by a
cluster of these processes.
This module manages both a queue of pending tasks, feeding processes pending
tasks when they are idle, as well as monitoring the child processes for errors
and crashes. Batch processes are also recycled after processing N tasks or
running for N seconds, in an effort to minimize the impact of any potential
memory leaks.
As of version 4, retry logic for tasks is a separate concern from this module.
This package powers [exiftool-vendored](https://photostructure.github.io/exiftool-vendored.js/),
whose source you can examine as an example consumer.
## Installation
Depending on your yarn/npm preference:
```bash
$ yarn add batch-cluster
# or
$ npm install --save batch-cluster
```
## Changelog
See [CHANGELOG.md](https://github.com/photostructure/batch-cluster.js/blob/main/CHANGELOG.md).
## Usage
The child process must use `stdin` and `stdout` for control/response.
BatchCluster will ensure a given process is only given one task at a time.
1. Create a singleton instance of
[BatchCluster](https://photostructure.github.io/batch-cluster.js/classes/BatchCluster.html).
Note the [constructor
options](https://photostructure.github.io/batch-cluster.js/classes/BatchCluster.html#constructor)
takes a union type of
- [ChildProcessFactory](https://photostructure.github.io/batch-cluster.js/interfaces/ChildProcessFactory.html)
and
- [BatchProcessOptions](https://photostructure.github.io/batch-cluster.js/interfaces/BatchProcessOptions.html),
both of which have no defaults, and
- [BatchClusterOptions](https://photostructure.github.io/batch-cluster.js/classes/BatchClusterOptions.html),
which has defaults that may or may not be relevant to your application.
1. The [default logger](https://photostructure.github.io/batch-cluster.js/interfaces/Logger.html)
writes warning and error messages to `console.warn` and `console.error`. You
can change this to your logger by using
[setLogger](https://photostructure.github.io/batch-cluster.js/modules.html#setLogger) or by providing a logger to the `BatchCluster` constructor.
1. Implement the [Parser](https://photostructure.github.io/batch-cluster.js/interfaces/Parser.html)
class to parse results from your child process.
1. Construct or extend the
[Task](https://photostructure.github.io/batch-cluster.js/classes/Task.html)
class with the desired command and the parser you built in the previous
step, and submit it to your BatchCluster's
[enqueueTask](https://photostructure.github.io/batch-cluster.js/classes/BatchCluster.html#enqueueTask)
method.
See
[src/test.ts](https://github.com/photostructure/batch-cluster.js/blob/main/src/test.ts)
for an example child process. Note that the script is _designed_ to be flaky on
order to test BatchCluster's retry and error handling code.
## Caution
The default `BatchClusterOptions.cleanupChildProcs` value of `true` means that BatchCluster will try to use `ps` to ensure Node's view of process state are correct, and that errant
processes are cleaned up.
If you run this in a docker image based off Alpine or Debian Slim, **this won't work properly unless you install the `procps` package.**
[See issue #13 for details.](https://github.com/photostructure/batch-cluster.js/issues/13)
+19
View File
@@ -0,0 +1,19 @@
# Security Policy
## Supported Versions
Only the latest version of this library is supported.
## Reporting a Vulnerability
If you find a vulnerability, *or even think you have*, please send an email to
the author. Each signed git release by `mceachen` contains a monitored email
address.
If you prefer, you may DM [@mrm](https://twitter.com/mrm) or
[@PhotoStructure](https://twitter.com/PhotoStructure).
Expect a reply in under 36 hours. Please try an alternative contact method if
you don't get a response.
Thank you!
+6
View File
@@ -0,0 +1,6 @@
/**
* Remove all elements from the given array that return false from the given
* predicate `filter`.
*/
export declare function filterInPlace<T>(arr: T[], filter: (t: T) => boolean): T[];
export declare function count<T>(arr: T[], predicate: (t: T, idx: number) => boolean): number;
+35
View File
@@ -0,0 +1,35 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.count = exports.filterInPlace = void 0;
/**
* Remove all elements from the given array that return false from the given
* predicate `filter`.
*/
function filterInPlace(arr, filter) {
const len = arr.length;
let j = 0;
// PERF: for-loop to avoid the additional closure from a forEach
for (let i = 0; i < len; i++) {
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
const ea = arr[i];
if (filter(ea)) {
if (i !== j)
arr[j] = ea;
j++;
}
}
arr.length = j;
return arr;
}
exports.filterInPlace = filterInPlace;
function count(arr, predicate) {
let acc = 0;
for (let idx = 0; idx < arr.length; idx++) {
// eslint-disable-next-line @typescript-eslint/no-non-null-assertion
if (predicate(arr[idx], idx))
acc++;
}
return acc;
}
exports.count = count;
//# sourceMappingURL=Array.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Array.js","sourceRoot":"","sources":["../src/Array.ts"],"names":[],"mappings":";;;AAAA;;;GAGG;AACH,SAAgB,aAAa,CAAI,GAAQ,EAAE,MAAyB;IAClE,MAAM,GAAG,GAAG,GAAG,CAAC,MAAM,CAAA;IACtB,IAAI,CAAC,GAAG,CAAC,CAAA;IACT,gEAAgE;IAChE,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC;QAC7B,oEAAoE;QACpE,MAAM,EAAE,GAAG,GAAG,CAAC,CAAC,CAAE,CAAA;QAClB,IAAI,MAAM,CAAC,EAAE,CAAC,EAAE,CAAC;YACf,IAAI,CAAC,KAAK,CAAC;gBAAE,GAAG,CAAC,CAAC,CAAC,GAAG,EAAE,CAAA;YACxB,CAAC,EAAE,CAAA;QACL,CAAC;IACH,CAAC;IACD,GAAG,CAAC,MAAM,GAAG,CAAC,CAAA;IACd,OAAO,GAAG,CAAA;AACZ,CAAC;AAdD,sCAcC;AAED,SAAgB,KAAK,CACnB,GAAQ,EACR,SAAyC;IAEzC,IAAI,GAAG,GAAG,CAAC,CAAA;IACX,KAAK,IAAI,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,GAAG,CAAC,MAAM,EAAE,GAAG,EAAE,EAAE,CAAC;QAC1C,oEAAoE;QACpE,IAAI,SAAS,CAAC,GAAG,CAAC,GAAG,CAAE,EAAE,GAAG,CAAC;YAAE,GAAG,EAAE,CAAA;IACtC,CAAC;IACD,OAAO,GAAG,CAAA;AACZ,CAAC;AAVD,sBAUC"}
+12
View File
@@ -0,0 +1,12 @@
export declare function delay(millis: number, unref?: boolean): Promise<void>;
/**
* Run the given thunk until the promise is resolved to true, or the timeout
* passes.
*/
export declare function until(f: (count: number) => boolean | Promise<boolean>, timeoutMs: number, delayMs?: number): Promise<boolean>;
/**
* @return a thunk that will call the underlying thunk at most every `minDelayMs`
* milliseconds. The thunk will accept a boolean, that, when set, will force the
* underlying thunk to be called (mostly useful for tests)
*/
export declare function ratelimit<T>(f: () => T, minDelayMs: number): () => T | undefined;
+53
View File
@@ -0,0 +1,53 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.ratelimit = exports.until = exports.delay = void 0;
const node_timers_1 = __importDefault(require("node:timers"));
function delay(millis, unref = false) {
return new Promise((resolve) => {
const t = node_timers_1.default.setTimeout(() => resolve(), millis);
if (unref)
t.unref();
});
}
exports.delay = delay;
/**
* Run the given thunk until the promise is resolved to true, or the timeout
* passes.
*/
async function until(f, timeoutMs, delayMs = 50) {
const timeoutAt = Date.now() + timeoutMs;
let count = 0;
while (Date.now() < timeoutAt) {
if (await f(count)) {
return true;
}
else {
count++;
await delay(delayMs);
}
}
return false;
}
exports.until = until;
/**
* @return a thunk that will call the underlying thunk at most every `minDelayMs`
* milliseconds. The thunk will accept a boolean, that, when set, will force the
* underlying thunk to be called (mostly useful for tests)
*/
function ratelimit(f, minDelayMs) {
let next = 0;
return (force) => {
if (Date.now() > next || force === true) {
next = Date.now() + minDelayMs;
return f();
}
else {
return;
}
};
}
exports.ratelimit = ratelimit;
//# sourceMappingURL=Async.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Async.js","sourceRoot":"","sources":["../src/Async.ts"],"names":[],"mappings":";;;;;;AAAA,8DAAgC;AAEhC,SAAgB,KAAK,CAAC,MAAc,EAAE,KAAK,GAAG,KAAK;IACjD,OAAO,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,EAAE;QACnC,MAAM,CAAC,GAAG,qBAAM,CAAC,UAAU,CAAC,GAAG,EAAE,CAAC,OAAO,EAAE,EAAE,MAAM,CAAC,CAAA;QACpD,IAAI,KAAK;YAAE,CAAC,CAAC,KAAK,EAAE,CAAA;IACtB,CAAC,CAAC,CAAA;AACJ,CAAC;AALD,sBAKC;AAED;;;GAGG;AACI,KAAK,UAAU,KAAK,CACzB,CAAgD,EAChD,SAAiB,EACjB,OAAO,GAAG,EAAE;IAEZ,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAA;IACxC,IAAI,KAAK,GAAG,CAAC,CAAA;IACb,OAAO,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,EAAE,CAAC;QAC9B,IAAI,MAAM,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC;YACnB,OAAO,IAAI,CAAA;QACb,CAAC;aAAM,CAAC;YACN,KAAK,EAAE,CAAA;YACP,MAAM,KAAK,CAAC,OAAO,CAAC,CAAA;QACtB,CAAC;IACH,CAAC;IACD,OAAO,KAAK,CAAA;AACd,CAAC;AAhBD,sBAgBC;AAED;;;;GAIG;AACH,SAAgB,SAAS,CACvB,CAAU,EACV,UAAkB;IAElB,IAAI,IAAI,GAAG,CAAC,CAAA;IACZ,OAAO,CAAC,KAAe,EAAE,EAAE;QACzB,IAAI,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;YACxC,IAAI,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,UAAU,CAAA;YAC9B,OAAO,CAAC,EAAE,CAAA;QACZ,CAAC;aAAM,CAAC;YACN,OAAM;QACR,CAAC;IACH,CAAC,CAAA;AACH,CAAC;AAbD,8BAaC"}
+176
View File
@@ -0,0 +1,176 @@
/// <reference types="node" />
import child_process from "node:child_process";
import { BatchClusterEmitter, BatchClusterEvents, ChildEndReason, TypedEventEmitter } from "./BatchClusterEmitter";
import { AllOpts, BatchClusterOptions } from "./BatchClusterOptions";
import { WhyNotHealthy, WhyNotReady } from "./BatchProcess";
import { BatchProcessOptions } from "./BatchProcessOptions";
import { Deferred } from "./Deferred";
import { Parser } from "./Parser";
import { Task } from "./Task";
export { BatchClusterOptions } from "./BatchClusterOptions";
export { BatchProcess } from "./BatchProcess";
export { Deferred } from "./Deferred";
export * from "./Logger";
export { SimpleParser } from "./Parser";
export { kill, pidExists, pids } from "./Pids";
export { Rate } from "./Rate";
export { Task } from "./Task";
export type { BatchClusterEmitter, BatchClusterEvents, BatchProcessOptions, ChildEndReason as ChildExitReason, Parser, TypedEventEmitter, WhyNotHealthy, WhyNotReady, };
/**
* These are required parameters for a given BatchCluster.
*/
export interface ChildProcessFactory {
/**
* Expected to be a simple call to execFile. Platform-specific code is the
* responsibility of this thunk. Error handlers will be registered as
* appropriate.
*
* If this function throws an error or rejects the promise _after_ you've
* spawned a child process, **the child process may continue to run** and leak
* system resources.
*/
readonly processFactory: () => child_process.ChildProcess | Promise<child_process.ChildProcess>;
}
/**
* BatchCluster instances manage 0 or more homogeneous child processes, and
* provide the main interface for enqueuing `Task`s via `enqueueTask`.
*
* Given the large number of configuration options, the constructor
* receives a single options hash. The most important of these are the
* `ChildProcessFactory`, which specifies the factory that creates
* ChildProcess instances, and `BatchProcessOptions`, which specifies how
* child tasks can be verified and shut down.
*/
export declare class BatchCluster {
#private;
readonly options: AllOpts;
readonly emitter: BatchClusterEmitter;
constructor(opts: Partial<BatchClusterOptions> & BatchProcessOptions & ChildProcessFactory);
/**
* @see BatchClusterEvents
*/
readonly on: <E extends keyof BatchClusterEvents>(eventName: E, listener: (...args: BatchClusterEvents[E] extends infer T ? T extends BatchClusterEvents[E] ? T extends (...args: infer A) => void ? A : never : never : never) => void) => BatchClusterEmitter;
/**
* @see BatchClusterEvents
* @since v9.0.0
*/
readonly off: <E extends keyof BatchClusterEvents>(eventName: E, listener: (...args: BatchClusterEvents[E] extends infer T ? T extends BatchClusterEvents[E] ? T extends (...args: infer A) => void ? A : never : never : never) => void) => BatchClusterEmitter;
get ended(): boolean;
/**
* Shut down this instance, and all child processes.
* @param gracefully should an attempt be made to finish in-flight tasks, or
* should we force-kill child PIDs.
*/
end(gracefully?: boolean): Deferred<void>;
/**
* Submits `task` for processing by a `BatchProcess` instance
*
* @return a Promise that is resolved or rejected once the task has been
* attempted on an idle BatchProcess
*/
enqueueTask<T>(task: Task<T>): Promise<T>;
/**
* @return true if all previously-enqueued tasks have settled
*/
get isIdle(): boolean;
/**
* @return the number of pending tasks
*/
get pendingTaskCount(): number;
/**
* @returns {number} the mean number of tasks completed by child processes
*/
get meanTasksPerProc(): number;
/**
* @return the total number of child processes created by this instance
*/
get spawnedProcCount(): number;
/**
* @return the current number of spawned child processes. Some (or all) may be idle.
*/
get procCount(): number;
/**
* @return the current number of child processes currently servicing tasks
*/
get busyProcCount(): number;
get startingProcCount(): number;
/**
* @return the current pending Tasks (mostly for testing)
*/
get pendingTasks(): Task<any>[];
/**
* @return the current running Tasks (mostly for testing)
*/
get currentTasks(): Task[];
/**
* For integration tests:
*/
get internalErrorCount(): number;
/**
* Verify that each BatchProcess PID is actually alive.
*
* @return the spawned PIDs that are still in the process table.
*/
pids(): number[];
/**
* For diagnostics. Contents may change.
*/
stats(): {
pendingTaskCount: number;
currentProcCount: number;
readyProcCount: number;
maxProcCount: number;
internalErrorCount: number;
startErrorRatePerMinute: number;
msBeforeNextSpawn: number;
spawnedProcCount: number;
childEndCounts: {
startError: number;
broken: number;
closed: number;
ending: number;
ended: number;
idle: number;
old: number;
"proc.close": number;
"proc.disconnect": number;
"proc.error": number;
"proc.exit": number;
"stderr.error": number;
stderr: number;
"stdin.error": number;
"stdout.error": number;
timeout: number;
tooMany: number;
unhealthy: number;
worn: number;
};
ending: boolean;
ended: boolean;
};
/**
* Get ended process counts (used for tests)
*/
countEndedChildProcs(why: ChildEndReason): number;
get childEndCounts(): {
[key in NonNullable<ChildEndReason>]: number;
};
/**
* Shut down any currently-running child processes. New child processes will
* be started automatically to handle new tasks.
*/
closeChildProcesses(gracefully?: boolean): Promise<void>;
/**
* Reset the maximum number of active child processes to `maxProcs`. Note that
* this is handled gracefully: child processes are only reduced as tasks are
* completed.
*/
setMaxProcs(maxProcs: number): void;
/**
* Run maintenance on currently spawned child processes. This method is
* normally invoked automatically as tasks are enqueued and processed.
*
* Only public for tests.
*/
vacuumProcs(): Promise<void[]>;
}
+473
View File
@@ -0,0 +1,473 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __exportStar = (this && this.__exportStar) || function(m, exports) {
for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _BatchCluster_instances, _BatchCluster_tasksPerProc, _BatchCluster_logger, _BatchCluster_procs, _BatchCluster_onIdleRequested, _BatchCluster_nextSpawnTime, _BatchCluster_lastPidsCheckTime, _BatchCluster_tasks, _BatchCluster_onIdleInterval, _BatchCluster_startErrorRate, _BatchCluster_spawnedProcs, _BatchCluster_endPromise, _BatchCluster_internalErrorCount, _BatchCluster_childEndCounts, _BatchCluster_beforeExitListener, _BatchCluster_exitListener, _BatchCluster_onIdleLater, _BatchCluster_onIdle, _BatchCluster_maybeCheckPids, _BatchCluster_execNextTask, _BatchCluster_maxSpawnDelay, _BatchCluster_procsToSpawn, _BatchCluster_maybeSpawnProcs, _BatchCluster_spawnNewProc;
Object.defineProperty(exports, "__esModule", { value: true });
exports.BatchCluster = exports.Task = exports.Rate = exports.pids = exports.pidExists = exports.kill = exports.SimpleParser = exports.Deferred = exports.BatchProcess = exports.BatchClusterOptions = void 0;
const node_events_1 = __importDefault(require("node:events"));
const node_process_1 = __importDefault(require("node:process"));
const node_timers_1 = __importDefault(require("node:timers"));
const Array_1 = require("./Array");
const BatchClusterOptions_1 = require("./BatchClusterOptions");
const BatchProcess_1 = require("./BatchProcess");
const Deferred_1 = require("./Deferred");
const Error_1 = require("./Error");
const Mean_1 = require("./Mean");
const Object_1 = require("./Object");
const Rate_1 = require("./Rate");
const String_1 = require("./String");
const Timeout_1 = require("./Timeout");
var BatchClusterOptions_2 = require("./BatchClusterOptions");
Object.defineProperty(exports, "BatchClusterOptions", { enumerable: true, get: function () { return BatchClusterOptions_2.BatchClusterOptions; } });
var BatchProcess_2 = require("./BatchProcess");
Object.defineProperty(exports, "BatchProcess", { enumerable: true, get: function () { return BatchProcess_2.BatchProcess; } });
var Deferred_2 = require("./Deferred");
Object.defineProperty(exports, "Deferred", { enumerable: true, get: function () { return Deferred_2.Deferred; } });
__exportStar(require("./Logger"), exports);
var Parser_1 = require("./Parser");
Object.defineProperty(exports, "SimpleParser", { enumerable: true, get: function () { return Parser_1.SimpleParser; } });
var Pids_1 = require("./Pids");
Object.defineProperty(exports, "kill", { enumerable: true, get: function () { return Pids_1.kill; } });
Object.defineProperty(exports, "pidExists", { enumerable: true, get: function () { return Pids_1.pidExists; } });
Object.defineProperty(exports, "pids", { enumerable: true, get: function () { return Pids_1.pids; } });
var Rate_2 = require("./Rate");
Object.defineProperty(exports, "Rate", { enumerable: true, get: function () { return Rate_2.Rate; } });
var Task_1 = require("./Task");
Object.defineProperty(exports, "Task", { enumerable: true, get: function () { return Task_1.Task; } });
/**
* BatchCluster instances manage 0 or more homogeneous child processes, and
* provide the main interface for enqueuing `Task`s via `enqueueTask`.
*
* Given the large number of configuration options, the constructor
* receives a single options hash. The most important of these are the
* `ChildProcessFactory`, which specifies the factory that creates
* ChildProcess instances, and `BatchProcessOptions`, which specifies how
* child tasks can be verified and shut down.
*/
class BatchCluster {
constructor(opts) {
_BatchCluster_instances.add(this);
_BatchCluster_tasksPerProc.set(this, new Mean_1.Mean());
_BatchCluster_logger.set(this, void 0);
_BatchCluster_procs.set(this, []);
_BatchCluster_onIdleRequested.set(this, false);
_BatchCluster_nextSpawnTime.set(this, 0);
_BatchCluster_lastPidsCheckTime.set(this, 0);
_BatchCluster_tasks.set(this, []);
_BatchCluster_onIdleInterval.set(this, void 0);
_BatchCluster_startErrorRate.set(this, new Rate_1.Rate());
_BatchCluster_spawnedProcs.set(this, 0);
_BatchCluster_endPromise.set(this, void 0);
_BatchCluster_internalErrorCount.set(this, 0);
_BatchCluster_childEndCounts.set(this, new Map());
this.emitter = new node_events_1.default.EventEmitter();
/**
* @see BatchClusterEvents
*/
this.on = this.emitter.on.bind(this.emitter);
/**
* @see BatchClusterEvents
* @since v9.0.0
*/
this.off = this.emitter.off.bind(this.emitter);
_BatchCluster_beforeExitListener.set(this, () => this.end(true));
_BatchCluster_exitListener.set(this, () => this.end(false));
_BatchCluster_onIdleLater.set(this, () => {
if (!__classPrivateFieldGet(this, _BatchCluster_onIdleRequested, "f")) {
__classPrivateFieldSet(this, _BatchCluster_onIdleRequested, true, "f");
node_timers_1.default.setTimeout(() => __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_onIdle).call(this), 1);
}
}
// NOT ASYNC: updates internal state:
);
this.options = (0, BatchClusterOptions_1.verifyOptions)({ ...opts, observer: this.emitter });
this.on("childEnd", (bp, why) => {
var _a;
__classPrivateFieldGet(this, _BatchCluster_tasksPerProc, "f").push(bp.taskCount);
__classPrivateFieldGet(this, _BatchCluster_childEndCounts, "f").set(why, ((_a = __classPrivateFieldGet(this, _BatchCluster_childEndCounts, "f").get(why)) !== null && _a !== void 0 ? _a : 0) + 1);
__classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f").call(this);
});
this.on("internalError", (error) => {
var _a;
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).error("BatchCluster: INTERNAL ERROR: " + error);
__classPrivateFieldSet(this, _BatchCluster_internalErrorCount, (_a = __classPrivateFieldGet(this, _BatchCluster_internalErrorCount, "f"), _a++, _a), "f");
});
this.on("noTaskData", (stdout, stderr, proc) => {
var _a;
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).warn("BatchCluster: child process emitted data with no current task. Consider setting streamFlushMillis to a higher value.", {
streamFlushMillis: this.options.streamFlushMillis,
stdout: (0, String_1.toS)(stdout),
stderr: (0, String_1.toS)(stderr),
proc_pid: proc === null || proc === void 0 ? void 0 : proc.pid,
});
__classPrivateFieldSet(this, _BatchCluster_internalErrorCount, (_a = __classPrivateFieldGet(this, _BatchCluster_internalErrorCount, "f"), _a++, _a), "f");
});
this.on("startError", (error) => {
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).warn("BatchCluster.onStartError(): " + error);
__classPrivateFieldGet(this, _BatchCluster_startErrorRate, "f").onEvent();
if (this.options.maxReasonableProcessFailuresPerMinute > 0 &&
__classPrivateFieldGet(this, _BatchCluster_startErrorRate, "f").eventsPerMinute >
this.options.maxReasonableProcessFailuresPerMinute) {
this.emitter.emit("fatalError", new Error(error +
"(start errors/min: " +
__classPrivateFieldGet(this, _BatchCluster_startErrorRate, "f").eventsPerMinute.toFixed(2) +
")"));
this.end();
}
else {
__classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f").call(this);
}
});
if (this.options.onIdleIntervalMillis > 0) {
__classPrivateFieldSet(this, _BatchCluster_onIdleInterval, node_timers_1.default.setInterval(() => __classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f").call(this), this.options.onIdleIntervalMillis), "f");
__classPrivateFieldGet(this, _BatchCluster_onIdleInterval, "f").unref(); // < don't prevent node from exiting
}
__classPrivateFieldSet(this, _BatchCluster_logger, this.options.logger, "f");
node_process_1.default.once("beforeExit", __classPrivateFieldGet(this, _BatchCluster_beforeExitListener, "f"));
node_process_1.default.once("exit", __classPrivateFieldGet(this, _BatchCluster_exitListener, "f"));
}
get ended() {
return __classPrivateFieldGet(this, _BatchCluster_endPromise, "f") != null;
}
/**
* Shut down this instance, and all child processes.
* @param gracefully should an attempt be made to finish in-flight tasks, or
* should we force-kill child PIDs.
*/
// NOT ASYNC so state transition happens immediately
end(gracefully = true) {
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).info("BatchCluster.end()", { gracefully });
if (__classPrivateFieldGet(this, _BatchCluster_endPromise, "f") == null) {
this.emitter.emit("beforeEnd");
(0, Object_1.map)(__classPrivateFieldGet(this, _BatchCluster_onIdleInterval, "f"), node_timers_1.default.clearInterval);
__classPrivateFieldSet(this, _BatchCluster_onIdleInterval, undefined, "f");
node_process_1.default.removeListener("beforeExit", __classPrivateFieldGet(this, _BatchCluster_beforeExitListener, "f"));
node_process_1.default.removeListener("exit", __classPrivateFieldGet(this, _BatchCluster_exitListener, "f"));
__classPrivateFieldSet(this, _BatchCluster_endPromise, new Deferred_1.Deferred().observe(this.closeChildProcesses(gracefully).then(() => {
this.emitter.emit("end");
})), "f");
}
return __classPrivateFieldGet(this, _BatchCluster_endPromise, "f");
}
/**
* Submits `task` for processing by a `BatchProcess` instance
*
* @return a Promise that is resolved or rejected once the task has been
* attempted on an idle BatchProcess
*/
enqueueTask(task) {
if (this.ended) {
task.reject(new Error("BatchCluster has ended, cannot enqueue " + task.command));
}
__classPrivateFieldGet(this, _BatchCluster_tasks, "f").push(task);
// Run #onIdle now (not later), to make sure the task gets enqueued asap if
// possible
__classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f").call(this);
// (BatchProcess will call our #onIdleLater when tasks settle or when they
// exit)
return task.promise;
}
/**
* @return true if all previously-enqueued tasks have settled
*/
get isIdle() {
return this.pendingTaskCount === 0 && this.busyProcCount === 0;
}
/**
* @return the number of pending tasks
*/
get pendingTaskCount() {
return __classPrivateFieldGet(this, _BatchCluster_tasks, "f").length;
}
/**
* @returns {number} the mean number of tasks completed by child processes
*/
get meanTasksPerProc() {
return __classPrivateFieldGet(this, _BatchCluster_tasksPerProc, "f").mean;
}
/**
* @return the total number of child processes created by this instance
*/
get spawnedProcCount() {
return __classPrivateFieldGet(this, _BatchCluster_spawnedProcs, "f");
}
/**
* @return the current number of spawned child processes. Some (or all) may be idle.
*/
get procCount() {
return __classPrivateFieldGet(this, _BatchCluster_procs, "f").length;
}
/**
* @return the current number of child processes currently servicing tasks
*/
get busyProcCount() {
return (0, Array_1.count)(__classPrivateFieldGet(this, _BatchCluster_procs, "f"),
// don't count procs that are starting up as "busy":
(ea) => !ea.starting && !ea.ending && !ea.idle);
}
get startingProcCount() {
return (0, Array_1.count)(__classPrivateFieldGet(this, _BatchCluster_procs, "f"),
// don't count procs that are starting up as "busy":
(ea) => ea.starting && !ea.ending);
}
/**
* @return the current pending Tasks (mostly for testing)
*/
get pendingTasks() {
return __classPrivateFieldGet(this, _BatchCluster_tasks, "f");
}
/**
* @return the current running Tasks (mostly for testing)
*/
get currentTasks() {
return __classPrivateFieldGet(this, _BatchCluster_procs, "f")
.map((ea) => ea.currentTask)
.filter((ea) => ea != null);
}
/**
* For integration tests:
*/
get internalErrorCount() {
return __classPrivateFieldGet(this, _BatchCluster_internalErrorCount, "f");
}
/**
* Verify that each BatchProcess PID is actually alive.
*
* @return the spawned PIDs that are still in the process table.
*/
pids() {
const arr = [];
for (const proc of [...__classPrivateFieldGet(this, _BatchCluster_procs, "f")]) {
if (proc != null && proc.running()) {
arr.push(proc.pid);
}
}
return arr;
}
/**
* For diagnostics. Contents may change.
*/
stats() {
var _a;
const readyProcCount = (0, Array_1.count)(__classPrivateFieldGet(this, _BatchCluster_procs, "f"), (ea) => ea.ready);
return {
pendingTaskCount: __classPrivateFieldGet(this, _BatchCluster_tasks, "f").length,
currentProcCount: __classPrivateFieldGet(this, _BatchCluster_procs, "f").length,
readyProcCount,
maxProcCount: this.options.maxProcs,
internalErrorCount: __classPrivateFieldGet(this, _BatchCluster_internalErrorCount, "f"),
startErrorRatePerMinute: __classPrivateFieldGet(this, _BatchCluster_startErrorRate, "f").eventsPerMinute,
msBeforeNextSpawn: Math.max(0, __classPrivateFieldGet(this, _BatchCluster_nextSpawnTime, "f") - Date.now()),
spawnedProcCount: this.spawnedProcCount,
childEndCounts: this.childEndCounts,
ending: __classPrivateFieldGet(this, _BatchCluster_endPromise, "f") != null,
ended: false === ((_a = __classPrivateFieldGet(this, _BatchCluster_endPromise, "f")) === null || _a === void 0 ? void 0 : _a.pending),
};
}
/**
* Get ended process counts (used for tests)
*/
countEndedChildProcs(why) {
var _a;
return (_a = __classPrivateFieldGet(this, _BatchCluster_childEndCounts, "f").get(why)) !== null && _a !== void 0 ? _a : 0;
}
get childEndCounts() {
return (0, Object_1.fromEntries)([...__classPrivateFieldGet(this, _BatchCluster_childEndCounts, "f").entries()]);
}
/**
* Shut down any currently-running child processes. New child processes will
* be started automatically to handle new tasks.
*/
async closeChildProcesses(gracefully = true) {
const procs = [...__classPrivateFieldGet(this, _BatchCluster_procs, "f")];
__classPrivateFieldGet(this, _BatchCluster_procs, "f").length = 0;
await Promise.all(procs.map((proc) => proc
.end(gracefully, "ending")
.catch((err) => this.emitter.emit("endError", (0, Error_1.asError)(err), proc))));
}
/**
* Reset the maximum number of active child processes to `maxProcs`. Note that
* this is handled gracefully: child processes are only reduced as tasks are
* completed.
*/
setMaxProcs(maxProcs) {
this.options.maxProcs = maxProcs;
// we may now be able to handle an enqueued task. Vacuum pids and see:
__classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f").call(this);
}
/**
* Run maintenance on currently spawned child processes. This method is
* normally invoked automatically as tasks are enqueued and processed.
*
* Only public for tests.
*/
// NOT ASYNC: updates internal state. only exported for tests.
vacuumProcs() {
__classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_maybeCheckPids).call(this);
const endPromises = [];
let pidsToReap = Math.max(0, __classPrivateFieldGet(this, _BatchCluster_procs, "f").length - this.options.maxProcs);
(0, Array_1.filterInPlace)(__classPrivateFieldGet(this, _BatchCluster_procs, "f"), (proc) => {
var _a;
// Only check `.idle` (not `.ready`) procs. We don't want to reap busy
// procs unless we're ending, and unhealthy procs (that we want to reap)
// won't be `.ready`.
if (proc.idle) {
// don't reap more than pidsToReap pids. We can't use #procs.length
// within filterInPlace because #procs.length only changes at iteration
// completion: the prior impl resulted in all idle pids getting reaped
// when maxProcs was reduced.
const why = (_a = proc.whyNotHealthy) !== null && _a !== void 0 ? _a : (--pidsToReap >= 0 ? "tooMany" : null);
if (why != null) {
endPromises.push(proc.end(true, why));
return false;
}
proc.maybeRunHealthcheck();
}
return true;
});
return Promise.all(endPromises);
}
}
exports.BatchCluster = BatchCluster;
_BatchCluster_tasksPerProc = new WeakMap(), _BatchCluster_logger = new WeakMap(), _BatchCluster_procs = new WeakMap(), _BatchCluster_onIdleRequested = new WeakMap(), _BatchCluster_nextSpawnTime = new WeakMap(), _BatchCluster_lastPidsCheckTime = new WeakMap(), _BatchCluster_tasks = new WeakMap(), _BatchCluster_onIdleInterval = new WeakMap(), _BatchCluster_startErrorRate = new WeakMap(), _BatchCluster_spawnedProcs = new WeakMap(), _BatchCluster_endPromise = new WeakMap(), _BatchCluster_internalErrorCount = new WeakMap(), _BatchCluster_childEndCounts = new WeakMap(), _BatchCluster_beforeExitListener = new WeakMap(), _BatchCluster_exitListener = new WeakMap(), _BatchCluster_onIdleLater = new WeakMap(), _BatchCluster_instances = new WeakSet(), _BatchCluster_onIdle = function _BatchCluster_onIdle() {
__classPrivateFieldSet(this, _BatchCluster_onIdleRequested, false, "f");
this.vacuumProcs();
while (__classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_execNextTask).call(this)) {
//
}
__classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_maybeSpawnProcs).call(this);
}, _BatchCluster_maybeCheckPids = function _BatchCluster_maybeCheckPids() {
if (this.options.cleanupChildProcs &&
this.options.pidCheckIntervalMillis > 0 &&
__classPrivateFieldGet(this, _BatchCluster_lastPidsCheckTime, "f") + this.options.pidCheckIntervalMillis < Date.now()) {
__classPrivateFieldSet(this, _BatchCluster_lastPidsCheckTime, Date.now(), "f");
void this.pids();
}
}, _BatchCluster_execNextTask = function _BatchCluster_execNextTask(retries = 1) {
if (__classPrivateFieldGet(this, _BatchCluster_tasks, "f").length === 0 || this.ended || retries < 0)
return false;
const readyProc = __classPrivateFieldGet(this, _BatchCluster_procs, "f").find((ea) => ea.ready);
// no procs are idle and healthy :(
if (readyProc == null) {
return false;
}
const task = __classPrivateFieldGet(this, _BatchCluster_tasks, "f").shift();
if (task == null) {
this.emitter.emit("internalError", new Error("unexpected null task"));
return false;
}
const submitted = readyProc.execTask(task);
if (!submitted) {
// This isn't an internal error: the proc may have needed to run a health
// check. Let's reschedule the task and try again:
__classPrivateFieldGet(this, _BatchCluster_tasks, "f").push(task);
// We don't want to return false here (it'll stop the onIdle loop) unless
// we actually can't submit the task:
return __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_execNextTask).call(this, retries--);
}
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).trace("BatchCluster.#execNextTask(): submitted task", {
child_pid: readyProc.pid,
task,
});
return submitted;
}, _BatchCluster_maxSpawnDelay = function _BatchCluster_maxSpawnDelay() {
// 10s delay is certainly long enough for .spawn() to return, even on a
// loaded windows machine.
return Math.max(10000, this.options.spawnTimeoutMillis);
}, _BatchCluster_procsToSpawn = function _BatchCluster_procsToSpawn() {
const remainingCapacity = this.options.maxProcs - __classPrivateFieldGet(this, _BatchCluster_procs, "f").length;
// take into account starting procs, so one task doesn't result in multiple
// processes being spawned:
const requestedCapacity = __classPrivateFieldGet(this, _BatchCluster_tasks, "f").length - this.startingProcCount;
const atLeast0 = Math.max(0, Math.min(remainingCapacity, requestedCapacity));
return this.options.minDelayBetweenSpawnMillis === 0
? // we can spin up multiple processes in parallel.
atLeast0
: // Don't spin up more than 1:
Math.min(1, atLeast0);
}, _BatchCluster_maybeSpawnProcs = async function _BatchCluster_maybeSpawnProcs() {
var _a;
let procsToSpawn = __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_procsToSpawn).call(this);
if (this.ended || __classPrivateFieldGet(this, _BatchCluster_nextSpawnTime, "f") > Date.now() || procsToSpawn === 0) {
return;
}
// prevent concurrent runs:
__classPrivateFieldSet(this, _BatchCluster_nextSpawnTime, Date.now() + __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_maxSpawnDelay).call(this), "f");
for (let i = 0; i < procsToSpawn; i++) {
if (this.ended) {
break;
}
// Kick the lock down the road:
__classPrivateFieldSet(this, _BatchCluster_nextSpawnTime, Date.now() + __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_maxSpawnDelay).call(this), "f");
__classPrivateFieldSet(this, _BatchCluster_spawnedProcs, (_a = __classPrivateFieldGet(this, _BatchCluster_spawnedProcs, "f"), _a++, _a), "f");
try {
const proc = __classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_spawnNewProc).call(this);
const result = await (0, Timeout_1.thenOrTimeout)(proc, this.options.spawnTimeoutMillis);
if (result === Timeout_1.Timeout) {
void proc
.then((bp) => {
void bp.end(false, "startError");
this.emitter.emit("startError", (0, Error_1.asError)("Failed to spawn process in " +
this.options.spawnTimeoutMillis +
"ms"), bp);
})
.catch((err) => {
// this should only happen if the processFactory throws a
// rejection:
this.emitter.emit("startError", (0, Error_1.asError)(err));
});
}
else {
__classPrivateFieldGet(this, _BatchCluster_logger, "f").call(this).debug("BatchCluster.#maybeSpawnProcs() started healthy child process", { pid: result.pid });
}
// tasks may have been popped off or setMaxProcs may have reduced
// maxProcs. Do this at the end so the for loop ends properly.
procsToSpawn = Math.min(__classPrivateFieldGet(this, _BatchCluster_instances, "m", _BatchCluster_procsToSpawn).call(this), procsToSpawn);
}
catch (err) {
this.emitter.emit("startError", (0, Error_1.asError)(err));
}
}
// YAY WE MADE IT.
// Only let more children get spawned after minDelay:
const delay = Math.max(100, this.options.minDelayBetweenSpawnMillis);
__classPrivateFieldSet(this, _BatchCluster_nextSpawnTime, Date.now() + delay, "f");
// And schedule #onIdle for that time:
node_timers_1.default.setTimeout(__classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f"), delay).unref();
}, _BatchCluster_spawnNewProc =
// must only be called by this.#maybeSpawnProcs()
async function _BatchCluster_spawnNewProc() {
// no matter how long it takes to spawn, always push the result into #procs
// so we don't leak child processes:
const proc = await this.options.processFactory();
const result = new BatchProcess_1.BatchProcess(proc, this.options, __classPrivateFieldGet(this, _BatchCluster_onIdleLater, "f"));
__classPrivateFieldGet(this, _BatchCluster_procs, "f").push(result);
return result;
};
//# sourceMappingURL=BatchCluster.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"BatchCluster.js","sourceRoot":"","sources":["../src/BatchCluster.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;AACA,8DAAgC;AAChC,gEAAkC;AAClC,8DAAgC;AAChC,mCAA8C;AAO9C,+DAI8B;AAC9B,iDAAyE;AAEzE,yCAAqC;AACrC,mCAAiC;AAEjC,iCAA6B;AAC7B,qCAA2C;AAE3C,iCAA6B;AAC7B,qCAA8B;AAE9B,uCAAkD;AAElD,6DAA2D;AAAlD,0HAAA,mBAAmB,OAAA;AAC5B,+CAA6C;AAApC,4GAAA,YAAY,OAAA;AACrB,uCAAqC;AAA5B,oGAAA,QAAQ,OAAA;AACjB,2CAAwB;AACxB,mCAAuC;AAA9B,sGAAA,YAAY,OAAA;AACrB,+BAA8C;AAArC,4FAAA,IAAI,OAAA;AAAE,iGAAA,SAAS,OAAA;AAAE,4FAAA,IAAI,OAAA;AAC9B,+BAA6B;AAApB,4FAAA,IAAI,OAAA;AACb,+BAA6B;AAApB,4FAAA,IAAI,OAAA;AA8Bb;;;;;;;;;GASG;AACH,MAAa,YAAY;IAiBvB,YACE,IAEqB;;QAnBd,qCAAgB,IAAI,WAAI,EAAE,EAAA;QAC1B,uCAAqB;QAErB,8BAAyB,EAAE,EAAA;QACpC,wCAAmB,KAAK,EAAA;QACxB,sCAAiB,CAAC,EAAA;QAClB,0CAAqB,CAAC,EAAA;QACb,8BAAiB,EAAE,EAAA;QAC5B,+CAA2C;QAClC,uCAAkB,IAAI,WAAI,EAAE,EAAA;QACrC,qCAAgB,CAAC,EAAA;QACjB,2CAA4B;QAC5B,2CAAsB,CAAC,EAAA;QACd,uCAAkB,IAAI,GAAG,EAA0B,EAAA;QACnD,YAAO,GAAG,IAAI,qBAAM,CAAC,YAAY,EAAyB,CAAA;QAqEnE;;WAEG;QACM,OAAE,GAAG,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;QAEhD;;;WAGG;QACM,QAAG,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,CAAA;QAEzC,2CAAsB,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,EAAA;QAC1C,qCAAgB,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,EAAA;QA6MrC,oCAAe,GAAG,EAAE;YAC3B,IAAI,CAAC,uBAAA,IAAI,qCAAiB,EAAE,CAAC;gBAC3B,uBAAA,IAAI,iCAAoB,IAAI,MAAA,CAAA;gBAC5B,qBAAM,CAAC,UAAU,CAAC,GAAG,EAAE,CAAC,uBAAA,IAAI,qDAAQ,MAAZ,IAAI,CAAU,EAAE,CAAC,CAAC,CAAA;YAC5C,CAAC;QACH,CAAC;QAED,qCAAqC;UAFpC;QA5RC,IAAI,CAAC,OAAO,GAAG,IAAA,mCAAa,EAAC,EAAE,GAAG,IAAI,EAAE,QAAQ,EAAE,IAAI,CAAC,OAAO,EAAE,CAAC,CAAA;QAEjE,IAAI,CAAC,EAAE,CAAC,UAAU,EAAE,CAAC,EAAE,EAAE,GAAG,EAAE,EAAE;;YAC9B,uBAAA,IAAI,kCAAc,CAAC,IAAI,CAAC,EAAE,CAAC,SAAS,CAAC,CAAA;YACrC,uBAAA,IAAI,oCAAgB,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,MAAA,uBAAA,IAAI,oCAAgB,CAAC,GAAG,CAAC,GAAG,CAAC,mCAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;YACvE,uBAAA,IAAI,iCAAa,MAAjB,IAAI,CAAe,CAAA;QACrB,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC,EAAE,CAAC,eAAe,EAAE,CAAC,KAAK,EAAE,EAAE;;YACjC,uBAAA,IAAI,4BAAQ,MAAZ,IAAI,CAAU,CAAC,KAAK,CAAC,gCAAgC,GAAG,KAAK,CAAC,CAAA;YAC9D,+DAAA,CAAA,wEAAwB,EAAxB,IAA0B,IAAA,CAAA,MAAA,CAAA;QAC5B,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC,EAAE,CAAC,YAAY,EAAE,CAAC,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE;;YAC7C,uBAAA,IAAI,4BAAQ,MAAZ,IAAI,CAAU,CAAC,IAAI,CACjB,sHAAsH,EACtH;gBACE,iBAAiB,EAAE,IAAI,CAAC,OAAO,CAAC,iBAAiB;gBACjD,MAAM,EAAE,IAAA,YAAG,EAAC,MAAM,CAAC;gBACnB,MAAM,EAAE,IAAA,YAAG,EAAC,MAAM,CAAC;gBACnB,QAAQ,EAAE,IAAI,aAAJ,IAAI,uBAAJ,IAAI,CAAE,GAAG;aACpB,CACF,CAAA;YACD,+DAAA,CAAA,wEAAwB,EAAxB,IAA0B,IAAA,CAAA,MAAA,CAAA;QAC5B,CAAC,CAAC,CAAA;QAEF,IAAI,CAAC,EAAE,CAAC,YAAY,EAAE,CAAC,KAAK,EAAE,EAAE;YAC9B,uBAAA,IAAI,4BAAQ,MAAZ,IAAI,CAAU,CAAC,IAAI,CAAC,+BAA+B,GAAG,KAAK,CAAC,CAAA;YAC5D,uBAAA,IAAI,oCAAgB,CAAC,OAAO,EAAE,CAAA;YAC9B,IACE,IAAI,CAAC,OAAO,CAAC,qCAAqC,GAAG,CAAC;gBACtD,uBAAA,IAAI,oCAAgB,CAAC,eAAe;oBAClC,IAAI,CAAC,OAAO,CAAC,qCAAqC,EACpD,CAAC;gBACD,IAAI,CAAC,OAAO,CAAC,IAAI,CACf,YAAY,EACZ,IAAI,KAAK,CACP,KAAK;oBACH,qBAAqB;oBACrB,uBAAA,IAAI,oCAAgB,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC,CAAC;oBAC/C,GAAG,CACN,CACF,CAAA;gBACD,IAAI,CAAC,GAAG,EAAE,CAAA;YACZ,CAAC;iBAAM,CAAC;gBACN,uBAAA,IAAI,iCAAa,MAAjB,IAAI,CAAe,CAAA;YACrB,CAAC;QACH,CAAC,CAAC,CAAA;QAEF,IAAI,IAAI,CAAC,OAAO,CAAC,oBAAoB,GAAG,CAAC,EAAE,CAAC;YAC1C,uBAAA,IAAI,gCAAmB,qBAAM,CAAC,WAAW,CACvC,GAAG,EAAE,CAAC,uBAAA,IAAI,iCAAa,MAAjB,IAAI,CAAe,EACzB,IAAI,CAAC,OAAO,CAAC,oBAAoB,CAClC,MAAA,CAAA;YACD,uBAAA,IAAI,oCAAgB,CAAC,KAAK,EAAE,CAAA,CAAC,oCAAoC;QACnE,CAAC;QACD,uBAAA,IAAI,wBAAW,IAAI,CAAC,OAAO,CAAC,MAAM,MAAA,CAAA;QAElC,sBAAO,CAAC,IAAI,CAAC,YAAY,EAAE,uBAAA,IAAI,wCAAoB,CAAC,CAAA;QACpD,sBAAO,CAAC,IAAI,CAAC,MAAM,EAAE,uBAAA,IAAI,kCAAc,CAAC,CAAA;IAC1C,CAAC;IAgBD,IAAI,KAAK;QACP,OAAO,uBAAA,IAAI,gCAAY,IAAI,IAAI,CAAA;IACjC,CAAC;IAED;;;;OAIG;IACH,oDAAoD;IACpD,GAAG,CAAC,UAAU,GAAG,IAAI;QACnB,uBAAA,IAAI,4BAAQ,MAAZ,IAAI,CAAU,CAAC,IAAI,CAAC,oBAAoB,EAAE,EAAE,UAAU,EAAE,CAAC,CAAA;QAEzD,IAAI,uBAAA,IAAI,gCAAY,IAAI,IAAI,EAAE,CAAC;YAC7B,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,WAAW,CAAC,CAAA;YAC9B,IAAA,YAAG,EAAC,uBAAA,IAAI,oCAAgB,EAAE,qBAAM,CAAC,aAAa,CAAC,CAAA;YAC/C,uBAAA,IAAI,gCAAmB,SAAS,MAAA,CAAA;YAChC,sBAAO,CAAC,cAAc,CAAC,YAAY,EAAE,uBAAA,IAAI,wCAAoB,CAAC,CAAA;YAC9D,sBAAO,CAAC,cAAc,CAAC,MAAM,EAAE,uBAAA,IAAI,kCAAc,CAAC,CAAA;YAClD,uBAAA,IAAI,4BAAe,IAAI,mBAAQ,EAAQ,CAAC,OAAO,CAC7C,IAAI,CAAC,mBAAmB,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,GAAG,EAAE;gBAC7C,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAA;YAC1B,CAAC,CAAC,CACH,MAAA,CAAA;QACH,CAAC;QAED,OAAO,uBAAA,IAAI,gCAAY,CAAA;IACzB,CAAC;IAED;;;;;OAKG;IACH,WAAW,CAAI,IAAa;QAC1B,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YACf,IAAI,CAAC,MAAM,CACT,IAAI,KAAK,CAAC,yCAAyC,GAAG,IAAI,CAAC,OAAO,CAAC,CACpE,CAAA;QACH,CAAC;QACD,uBAAA,IAAI,2BAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;QAEtB,2EAA2E;QAC3E,WAAW;QACX,uBAAA,IAAI,iCAAa,MAAjB,IAAI,CAAe,CAAA;QAEnB,0EAA0E;QAC1E,QAAQ;QAER,OAAO,IAAI,CAAC,OAAO,CAAA;IACrB,CAAC;IAED;;OAEG;IACH,IAAI,MAAM;QACR,OAAO,IAAI,CAAC,gBAAgB,KAAK,CAAC,IAAI,IAAI,CAAC,aAAa,KAAK,CAAC,CAAA;IAChE,CAAC;IAED;;OAEG;IACH,IAAI,gBAAgB;QAClB,OAAO,uBAAA,IAALine truncated
+105
View File
@@ -0,0 +1,105 @@
/// <reference types="node" />
import { BatchProcess, WhyNotHealthy } from "./BatchProcess";
import { Task } from "./Task";
type Args<T> = T extends (...args: infer A) => void ? A : never;
export type ChildEndReason = WhyNotHealthy | "tooMany";
export interface TypedEventEmitter<T> {
once<E extends keyof T>(eventName: E, listener: (...args: Args<T[E]>) => void): this;
on<E extends keyof T>(eventName: E, listener: (...args: Args<T[E]>) => void): this;
off<E extends keyof T>(eventName: E, listener: (...args: Args<T[E]>) => void): this;
emit<E extends keyof T>(eventName: E, ...args: Args<T[E]>): boolean;
listeners<E extends keyof T>(event: E): Function[];
removeAllListeners(eventName?: keyof T): this;
}
/**
* This interface describes the BatchCluster's event names as fields. The type
* of the field describes the event data payload.
*
* See {@link BatchClusterEmitter} for more details.
*/
export interface BatchClusterEvents {
/**
* Emitted when a child process has started
*/
childStart: (childProcess: BatchProcess) => void;
/**
* Emitted when a child process has ended
*/
childEnd: (childProcess: BatchProcess, reason: ChildEndReason) => void;
/**
* Emitted when a child process fails to spin up and run the {@link BatchProcessOptions.versionCommand} successfully within {@link BatchClusterOptions.spawnTimeoutMillis}.
*
* @param childProcess will be undefined if the error is from {@link ChildProcessFactory.processFactory}
*/
startError: (error: Error, childProcess?: BatchProcess) => void;
/**
* Emitted when an internal consistency check fails
*/
internalError: (error: Error) => void;
/**
* Emitted when `.end()` is called because the error rate has exceeded
* {@link BatchClusterOptions.maxReasonableProcessFailuresPerMinute}
*/
fatalError: (error: Error) => void;
/**
* Emitted when tasks receive data, which may be partial chunks from the task
* stream.
*/
taskData: (data: Buffer | string, task: Task | undefined, proc: BatchProcess) => void;
/**
* Emitted when a task has been resolved
*/
taskResolved: (task: Task, proc: BatchProcess) => void;
/**
* Emitted when a task times out. Note that a `taskError` event always succeeds these events.
*/
taskTimeout: (timeoutMs: number, task: Task, proc: BatchProcess) => void;
/**
* Emitted when a task has an error
*/
taskError: (error: Error, task: Task, proc: BatchProcess) => void;
/**
* Emitted when child processes write to stdout or stderr without a current
* task
*/
noTaskData: (stdoutData: string | Buffer | null, stderrData: string | Buffer | null, proc: BatchProcess) => void;
/**
* Emitted when a process fails health checks
*/
healthCheckError: (error: Error, proc: BatchProcess) => void;
/**
* Emitted when a child process has an error during shutdown
*/
endError: (error: Error, proc?: BatchProcess) => void;
/**
* Emitted when this instance is in the process of ending.
*/
beforeEnd: () => void;
/**
* Emitted when this instance has ended. No child processes should remain at
* this point.
*/
end: () => void;
}
/**
* The BatchClusterEmitter signature is built up automatically by the
* {@link BatchClusterEvents} interface, which ensures `.on`, `.off`, and
* `.emit` signatures are all consistent, and include the correct data payloads
* for all of BatchCluster's events.
*
* This approach has some benefits:
*
* - it ensures that on(), off(), and emit() signatures are all consistent,
* - supports editor autocomplete, and
* - offers strong typing,
*
* but has one drawback:
*
* - jsdocs don't list all signatures directly: you have to visit the event
* source interface.
*
* See {@link BatchClusterEvents} for a the list of events and their payload
* signatures
*/
export type BatchClusterEmitter = TypedEventEmitter<BatchClusterEvents>;
export {};
+3
View File
@@ -0,0 +1,3 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
//# sourceMappingURL=BatchClusterEmitter.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"BatchClusterEmitter.js","sourceRoot":"","sources":["../src/BatchClusterEmitter.ts"],"names":[],"mappings":""}
+163
View File
@@ -0,0 +1,163 @@
import { ChildProcessFactory } from "./BatchCluster";
import { BatchClusterEmitter } from "./BatchClusterEmitter";
import { BatchProcessOptions } from "./BatchProcessOptions";
import { InternalBatchProcessOptions } from "./InternalBatchProcessOptions";
import { Logger } from "./Logger";
export declare const secondMs = 1000;
export declare const minuteMs: number;
/**
* These parameter values have somewhat sensible defaults, but can be
* overridden for a given BatchCluster.
*/
export declare class BatchClusterOptions {
/**
* No more than `maxProcs` child processes will be run at a given time
* to serve pending tasks.
*
* Defaults to 1.
*/
maxProcs: number;
/**
* Child processes will be recycled when they reach this age.
*
* This value must not be less than `spawnTimeoutMillis` or
* `taskTimeoutMillis`.
*
* Defaults to 5 minutes. Set to 0 to disable.
*/
maxProcAgeMillis: number;
/**
* This is the minimum interval between calls to BatchCluster's #onIdle
* method, which runs general janitorial processes like child process
* management and task queue validation.
*
* Must be &gt; 0. Defaults to 10 seconds.
*/
onIdleIntervalMillis: number;
/**
* If the initial `versionCommand` fails for new spawned processes more
* than this rate, end this BatchCluster and throw an error, because
* something is terribly wrong.
*
* If this backstop didn't exist, new (failing) child processes would be
* created indefinitely.
*
* Defaults to 10. Set to 0 to disable.
*/
maxReasonableProcessFailuresPerMinute: number;
/**
* Spawning new child processes and servicing a "version" task must not take
* longer than `spawnTimeoutMillis` before the process is considered failed,
* and need to be restarted. Be pessimistic here--windows can regularly take
* several seconds to spin up a process, thanks to antivirus shenanigans.
*
* Defaults to 15 seconds. Set to 0 to disable.
*/
spawnTimeoutMillis: number;
/**
* If maxProcs &gt; 1, spawning new child processes to process tasks can slow
* down initial processing, and create unnecessary processes.
*
* Must be &gt;= 0ms. Defaults to 1.5 seconds.
*/
minDelayBetweenSpawnMillis: number;
/**
* If commands take longer than this, presume the underlying process is dead
* and we should fail the task.
*
* This should be set to something on the order of seconds.
*
* Defaults to 10 seconds. Set to 0 to disable.
*/
taskTimeoutMillis: number;
/**
* Processes will be recycled after processing `maxTasksPerProcess` tasks.
* Depending on the commands and platform, batch mode commands shouldn't
* exhibit unduly memory leaks for at least tens if not hundreds of tasks.
* Setting this to a low number (like less than 10) will impact performance
* markedly, due to OS process start/stop maintenance. Setting this to a very
* high number (> 1000) may result in more memory being consumed than
* necessary.
*
* Must be &gt;= 0. Defaults to 500
*/
maxTasksPerProcess: number;
/**
* When `this.end()` is called, or Node broadcasts the `beforeExit` event,
* this is the milliseconds spent waiting for currently running tasks to
* finish before sending kill signals to child processes.
*
* Setting this value to 0 means child processes will immediately receive a
* kill signal to shut down. Any pending requests may be interrupted. Must be
* &gt;= 0. Defaults to 500ms.
*/
endGracefulWaitTimeMillis: number;
/**
* When a task sees a "pass" or "fail" from either stdout or stderr, it needs
* to wait for the other stream to finish flushing to ensure the task's Parser
* sees the entire relevant stream contents. A larger number may be required
* for slower computers to prevent internal errors due to lack of stream
* coercion.
*
* Note that this puts a hard lower limit on task latency, so don't set this
* to a large number: no task will resolve faster than this value (in millis).
*
* If you set this value too low, tasks may be erroneously resolved or
* rejected (depending on which stream is handled first).
*
* Your system may support a smaller value: this is a pessimistic default. If
* this is set too low, you'll see `noTaskData` events.
*
* Setting this to 0 makes whatever flushes first--stdout and stderr--and will
* most likely result in internal errors (due to stream buffers not being able
* to be associated to tasks that were just settled)
*/
streamFlushMillis: number;
/**
* Should batch-cluster try to clean up after spawned processes that don't
* shut down?
*
* Only disable this if you have another means of PID cleanup.
*
* Defaults to `true`.
*/
cleanupChildProcs: boolean;
/**
* If a child process is idle for more than this value (in milliseconds), shut
* it down to reduce system resource consumption.
*
* A value of ~10 seconds to a couple minutes would be reasonable. Set this to
* 0 to disable this feature.
*/
maxIdleMsPerProcess: number;
/**
* How many failed tasks should a process be allowed to process before it is
* recycled?
*
* Set this to 0 to disable this feature.
*/
maxFailedTasksPerProcess: number;
/**
* If `healthCheckCommand` is set, how frequently should we check for
* unhealthy child processes?
*
* Set this to 0 to disable this feature.
*/
healthCheckIntervalMillis: number;
/**
* Verify child processes are still running by checking the OS process table.
*
* Set this to 0 to disable this feature.
*/
pidCheckIntervalMillis: number;
/**
* A BatchCluster instance and associated BatchProcess instances will share
* this `Logger`. Defaults to the `Logger` instance provided to `setLogger()`.
*/
logger: () => Logger;
}
export interface WithObserver {
observer: BatchClusterEmitter;
}
export type AllOpts = BatchClusterOptions & InternalBatchProcessOptions & ChildProcessFactory & WithObserver;
export declare function verifyOptions(opts: Partial<BatchClusterOptions> & BatchProcessOptions & ChildProcessFactory & WithObserver): AllOpts;
+216
View File
@@ -0,0 +1,216 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.verifyOptions = exports.BatchClusterOptions = exports.minuteMs = exports.secondMs = void 0;
const Logger_1 = require("./Logger");
const Platform_1 = require("./Platform");
const String_1 = require("./String");
exports.secondMs = 1000;
exports.minuteMs = 60 * exports.secondMs;
/**
* These parameter values have somewhat sensible defaults, but can be
* overridden for a given BatchCluster.
*/
class BatchClusterOptions {
constructor() {
/**
* No more than `maxProcs` child processes will be run at a given time
* to serve pending tasks.
*
* Defaults to 1.
*/
this.maxProcs = 1;
/**
* Child processes will be recycled when they reach this age.
*
* This value must not be less than `spawnTimeoutMillis` or
* `taskTimeoutMillis`.
*
* Defaults to 5 minutes. Set to 0 to disable.
*/
this.maxProcAgeMillis = 5 * exports.minuteMs;
/**
* This is the minimum interval between calls to BatchCluster's #onIdle
* method, which runs general janitorial processes like child process
* management and task queue validation.
*
* Must be &gt; 0. Defaults to 10 seconds.
*/
this.onIdleIntervalMillis = 10 * exports.secondMs;
/**
* If the initial `versionCommand` fails for new spawned processes more
* than this rate, end this BatchCluster and throw an error, because
* something is terribly wrong.
*
* If this backstop didn't exist, new (failing) child processes would be
* created indefinitely.
*
* Defaults to 10. Set to 0 to disable.
*/
this.maxReasonableProcessFailuresPerMinute = 10;
/**
* Spawning new child processes and servicing a "version" task must not take
* longer than `spawnTimeoutMillis` before the process is considered failed,
* and need to be restarted. Be pessimistic here--windows can regularly take
* several seconds to spin up a process, thanks to antivirus shenanigans.
*
* Defaults to 15 seconds. Set to 0 to disable.
*/
this.spawnTimeoutMillis = 15 * exports.secondMs;
/**
* If maxProcs &gt; 1, spawning new child processes to process tasks can slow
* down initial processing, and create unnecessary processes.
*
* Must be &gt;= 0ms. Defaults to 1.5 seconds.
*/
this.minDelayBetweenSpawnMillis = 1.5 * exports.secondMs;
/**
* If commands take longer than this, presume the underlying process is dead
* and we should fail the task.
*
* This should be set to something on the order of seconds.
*
* Defaults to 10 seconds. Set to 0 to disable.
*/
this.taskTimeoutMillis = 10 * exports.secondMs;
/**
* Processes will be recycled after processing `maxTasksPerProcess` tasks.
* Depending on the commands and platform, batch mode commands shouldn't
* exhibit unduly memory leaks for at least tens if not hundreds of tasks.
* Setting this to a low number (like less than 10) will impact performance
* markedly, due to OS process start/stop maintenance. Setting this to a very
* high number (> 1000) may result in more memory being consumed than
* necessary.
*
* Must be &gt;= 0. Defaults to 500
*/
this.maxTasksPerProcess = 500;
/**
* When `this.end()` is called, or Node broadcasts the `beforeExit` event,
* this is the milliseconds spent waiting for currently running tasks to
* finish before sending kill signals to child processes.
*
* Setting this value to 0 means child processes will immediately receive a
* kill signal to shut down. Any pending requests may be interrupted. Must be
* &gt;= 0. Defaults to 500ms.
*/
this.endGracefulWaitTimeMillis = 500;
/**
* When a task sees a "pass" or "fail" from either stdout or stderr, it needs
* to wait for the other stream to finish flushing to ensure the task's Parser
* sees the entire relevant stream contents. A larger number may be required
* for slower computers to prevent internal errors due to lack of stream
* coercion.
*
* Note that this puts a hard lower limit on task latency, so don't set this
* to a large number: no task will resolve faster than this value (in millis).
*
* If you set this value too low, tasks may be erroneously resolved or
* rejected (depending on which stream is handled first).
*
* Your system may support a smaller value: this is a pessimistic default. If
* this is set too low, you'll see `noTaskData` events.
*
* Setting this to 0 makes whatever flushes first--stdout and stderr--and will
* most likely result in internal errors (due to stream buffers not being able
* to be associated to tasks that were just settled)
*/
// These values were found by trial and error using GitHub CI boxes, which
// should be the bottom of the barrel, performance-wise, of any computer.
this.streamFlushMillis = Platform_1.isMac ? 100 : Platform_1.isWin ? 200 : 30;
/**
* Should batch-cluster try to clean up after spawned processes that don't
* shut down?
*
* Only disable this if you have another means of PID cleanup.
*
* Defaults to `true`.
*/
this.cleanupChildProcs = true;
/**
* If a child process is idle for more than this value (in milliseconds), shut
* it down to reduce system resource consumption.
*
* A value of ~10 seconds to a couple minutes would be reasonable. Set this to
* 0 to disable this feature.
*/
this.maxIdleMsPerProcess = 0;
/**
* How many failed tasks should a process be allowed to process before it is
* recycled?
*
* Set this to 0 to disable this feature.
*/
this.maxFailedTasksPerProcess = 2;
/**
* If `healthCheckCommand` is set, how frequently should we check for
* unhealthy child processes?
*
* Set this to 0 to disable this feature.
*/
this.healthCheckIntervalMillis = 0;
/**
* Verify child processes are still running by checking the OS process table.
*
* Set this to 0 to disable this feature.
*/
this.pidCheckIntervalMillis = 2 * exports.minuteMs;
/**
* A BatchCluster instance and associated BatchProcess instances will share
* this `Logger`. Defaults to the `Logger` instance provided to `setLogger()`.
*/
this.logger = Logger_1.logger;
}
}
exports.BatchClusterOptions = BatchClusterOptions;
function escapeRegExp(s) {
return (0, String_1.toS)(s).replace(/[-.,\\^$*+?()|[\]{}]/g, "\\$&");
}
function toRe(s) {
return s instanceof RegExp
? s
: new RegExp("(?:\\n|^)" + escapeRegExp(s) + "(?:\\r?\\n|$)");
}
function verifyOptions(opts) {
const result = {
...new BatchClusterOptions(),
...opts,
passRE: toRe(opts.pass),
failRE: toRe(opts.fail),
};
const errors = [];
function notBlank(fieldName) {
const v = (0, String_1.toS)(result[fieldName]);
if ((0, String_1.blank)(v)) {
errors.push(fieldName + " must not be blank");
}
}
function gte(fieldName, value, why) {
const v = result[fieldName];
if (v < value) {
const msg = `${fieldName} must be greater than or equal to ${value}${(0, String_1.blank)(why) ? "" : ": " + why}`;
errors.push(msg);
}
}
notBlank("versionCommand");
notBlank("pass");
notBlank("fail");
gte("maxTasksPerProcess", 1);
gte("maxProcs", 1);
if (opts.maxProcAgeMillis != null &&
opts.maxProcAgeMillis > 0 &&
result.taskTimeoutMillis) {
gte("maxProcAgeMillis", Math.max(result.spawnTimeoutMillis, result.taskTimeoutMillis), `the max value of spawnTimeoutMillis (${result.spawnTimeoutMillis}) and taskTimeoutMillis (${result.taskTimeoutMillis})`);
}
// 0 disables:
gte("minDelayBetweenSpawnMillis", 0);
gte("onIdleIntervalMillis", 0);
gte("endGracefulWaitTimeMillis", 0);
gte("maxReasonableProcessFailuresPerMinute", 0);
gte("streamFlushMillis", 0);
if (errors.length > 0) {
throw new Error("BatchCluster was given invalid options: " + errors.join("; "));
}
return result;
}
exports.verifyOptions = verifyOptions;
//# sourceMappingURL=BatchClusterOptions.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"BatchClusterOptions.js","sourceRoot":"","sources":["../src/BatchClusterOptions.ts"],"names":[],"mappings":";;;AAIA,qCAAyC;AACzC,yCAAyC;AACzC,qCAAqC;AAExB,QAAA,QAAQ,GAAG,IAAI,CAAA;AACf,QAAA,QAAQ,GAAG,EAAE,GAAG,gBAAQ,CAAA;AAErC;;;GAGG;AACH,MAAa,mBAAmB;IAAhC;QACE;;;;;WAKG;QACH,aAAQ,GAAG,CAAC,CAAA;QAEZ;;;;;;;WAOG;QACH,qBAAgB,GAAG,CAAC,GAAG,gBAAQ,CAAA;QAE/B;;;;;;WAMG;QACH,yBAAoB,GAAG,EAAE,GAAG,gBAAQ,CAAA;QAEpC;;;;;;;;;WASG;QACH,0CAAqC,GAAG,EAAE,CAAA;QAE1C;;;;;;;WAOG;QACH,uBAAkB,GAAG,EAAE,GAAG,gBAAQ,CAAA;QAElC;;;;;WAKG;QACH,+BAA0B,GAAG,GAAG,GAAG,gBAAQ,CAAA;QAE3C;;;;;;;WAOG;QACH,sBAAiB,GAAG,EAAE,GAAG,gBAAQ,CAAA;QAEjC;;;;;;;;;;WAUG;QACH,uBAAkB,GAAG,GAAG,CAAA;QAExB;;;;;;;;WAQG;QACH,8BAAyB,GAAG,GAAG,CAAA;QAE/B;;;;;;;;;;;;;;;;;;;WAmBG;QACH,0EAA0E;QAC1E,yEAAyE;QACzE,sBAAiB,GAAG,gBAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,gBAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAA;QAElD;;;;;;;WAOG;QACH,sBAAiB,GAAG,IAAI,CAAA;QAExB;;;;;;WAMG;QACH,wBAAmB,GAAG,CAAC,CAAA;QAEvB;;;;;WAKG;QACH,6BAAwB,GAAG,CAAC,CAAA;QAE5B;;;;;WAKG;QACH,8BAAyB,GAAG,CAAC,CAAA;QAE7B;;;;WAIG;QACH,2BAAsB,GAAG,CAAC,GAAG,gBAAQ,CAAA;QAErC;;;WAGG;QACH,WAAM,GAAiB,eAAM,CAAA;IAC/B,CAAC;CAAA;AAnKD,kDAmKC;AAWD,SAAS,YAAY,CAAC,CAAS;IAC7B,OAAO,IAAA,YAAG,EAAC,CAAC,CAAC,CAAC,OAAO,CAAC,uBAAuB,EAAE,MAAM,CAAC,CAAA;AACxD,CAAC;AAED,SAAS,IAAI,CAAC,CAAkB;IAC9B,OAAO,CAAC,YAAY,MAAM;QACxB,CAAC,CAAC,CAAC;QACH,CAAC,CAAC,IAAI,MAAM,CAAC,WAAW,GAAG,YAAY,CAAC,CAAC,CAAC,GAAG,eAAe,CAAC,CAAA;AACjE,CAAC;AAED,SAAgB,aAAa,CAC3B,IAGc;IAEd,MAAM,MAAM,GAAG;QACb,GAAG,IAAI,mBAAmB,EAAE;QAC5B,GAAG,IAAI;QACP,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;QACvB,MAAM,EAAE,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC;KACxB,CAAA;IAED,MAAM,MAAM,GAAa,EAAE,CAAA;IAE3B,SAAS,QAAQ,CAAC,SAAwB;QACxC,MAAM,CAAC,GAAG,IAAA,YAAG,EAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CAAA;QAChC,IAAI,IAAA,cAAK,EAAC,CAAC,CAAC,EAAE,CAAC;YACb,MAAM,CAAC,IAAI,CAAC,SAAS,GAAG,oBAAoB,CAAC,CAAA;QAC/C,CAAC;IACH,CAAC;IAED,SAAS,GAAG,CAAC,SAAwB,EAAE,KAAa,EAAE,GAAY;QAChE,MAAM,CAAC,GAAG,MAAM,CAAC,SAAS,CAAW,CAAA;QACrC,IAAI,CAAC,GAAG,KAAK,EAAE,CAAC;YACd,MAAM,GAAG,GAAG,GAAG,SAAS,qCAAqC,KAAK,GAAG,IAAA,cAAK,EAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,GAAG,GAAG,EAAE,CAAA;YACnG,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;QAClB,CAAC;IACH,CAAC;IAED,QAAQ,CAAC,gBAAgB,CAAC,CAAA;IAC1B,QAAQ,CAAC,MAAM,CAAC,CAAA;IAChB,QAAQ,CAAC,MAAM,CAAC,CAAA;IAEhB,GAAG,CAAC,oBAAoB,EAAE,CAAC,CAAC,CAAA;IAE5B,GAAG,CAAC,UAAU,EAAE,CAAC,CAAC,CAAA;IAElB,IACE,IAAI,CAAC,gBAAgB,IAAI,IAAI;QAC7B,IAAI,CAAC,gBAAgB,GAAG,CAAC;QACzB,MAAM,CAAC,iBAAiB,EACxB,CAAC;QACD,GAAG,CACD,kBAAkB,EAClB,IAAI,CAAC,GAAG,CAAC,MAAM,CAAC,kBAAkB,EAAE,MAAM,CAAC,iBAAiB,CAAC,EAC7D,wCAAwC,MAAM,CAAC,kBAAkB,4BAA4B,MAAM,CAAC,iBAAiB,GAAG,CACzH,CAAA;IACH,CAAC;IACD,cAAc;IACd,GAAG,CAAC,4BAA4B,EAAE,CAAC,CAAC,CAAA;IACpC,GAAG,CAAC,sBAAsB,EAAE,CAAC,CAAC,CAAA;IAC9B,GAAG,CAAC,2BAA2B,EAAE,CAAC,CAAC,CAAA;IACnC,GAAG,CAAC,uCAAuC,EAAE,CAAC,CAAC,CAAA;IAC/C,GAAG,CAAC,mBAAmB,EAAE,CAAC,CAAC,CAAA;IAE3B,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QACtB,MAAM,IAAI,KAAK,CACb,0CAA0C,GAAG,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAC/D,CAAA;IACH,CAAC;IAED,OAAO,MAAM,CAAA;AACf,CAAC;AA/DD,sCA+DC"}
+92
View File
@@ -0,0 +1,92 @@
/// <reference types="node" />
import child_process from "node:child_process";
import { InternalBatchProcessOptions } from "./InternalBatchProcessOptions";
import { Task } from "./Task";
export type WhyNotHealthy = "broken" | "closed" | "ending" | "ended" | "idle" | "old" | "proc.close" | "proc.disconnect" | "proc.error" | "proc.exit" | "stderr.error" | "stderr" | "stdin.error" | "stdout.error" | "timeout" | "tooMany" | "startError" | "unhealthy" | "worn";
export type WhyNotReady = WhyNotHealthy | "busy";
/**
* BatchProcess manages the care and feeding of a single child process.
*/
export declare class BatchProcess {
#private;
readonly proc: child_process.ChildProcess;
readonly opts: InternalBatchProcessOptions;
private readonly onIdle;
readonly name: string;
readonly pid: number;
readonly start: number;
readonly startupTaskId: number;
failedTaskCount: number;
/**
* @param onIdle to be called when internal state changes (like the current
* task is resolved, or the process exits)
*/
constructor(proc: child_process.ChildProcess, opts: InternalBatchProcessOptions, onIdle: () => void);
get currentTask(): Task | undefined;
get taskCount(): number;
get starting(): boolean;
/**
* @return true if `this.end()` has been requested (which may be due to the
* child process exiting)
*/
get ending(): boolean;
/**
* @return true if `this.end()` has completed running, which includes child
* process cleanup. Note that this may return `true` and the process table may
* still include the child pid. Call {@link BatchProcess#running()} for an authoritative
* (but expensive!) answer.
*/
get ended(): boolean;
/**
* @return true if the child process has exited and is no longer in the
* process table. Note that this may be erroneously false if the process table
* hasn't been checked. Call {@link BatchProcess#running()} for an authoritative (but
* expensive!) answer.
*/
get exited(): boolean;
/**
* @return a string describing why this process should be recycled, or null if
* the process passes all health checks. Note that this doesn't include if
* we're already busy: see {@link BatchProcess.whyNotReady} if you need to
* know if a process can handle a new task.
*/
get whyNotHealthy(): WhyNotHealthy | null;
/**
* @return true if the process doesn't need to be recycled.
*/
get healthy(): boolean;
/**
* @return true iff no current task. Does not take into consideration if the
* process has ended or should be recycled: see {@link BatchProcess.ready}.
*/
get idle(): boolean;
/**
* @return a string describing why this process cannot currently handle a new
* task, or `undefined` if this process is idle and healthy.
*/
get whyNotReady(): WhyNotReady | null;
/**
* @return true iff this process is both healthy and idle, and ready for a
* new task.
*/
get ready(): boolean;
get idleMs(): number;
/**
* @return true if the child process is in the process table
*/
running(): boolean;
notRunning(): boolean;
maybeRunHealthcheck(): Task | undefined;
execTask(task: Task): boolean;
/**
* End this child process.
*
* @param gracefully Wait for any current task to be resolved or rejected
* before shutting down the child process.
* @param reason who called end() (used for logging)
* @return Promise that will be resolved when the process has completed.
* Subsequent calls to end() will ignore the parameters and return the first
* endPromise.
*/
end(gracefully: boolean | undefined, reason: WhyNotHealthy): Promise<void>;
}
+504
View File
@@ -0,0 +1,504 @@
"use strict";
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _BatchProcess_instances, _BatchProcess_lastHealthCheck, _BatchProcess_healthCheckFailures, _BatchProcess_logger, _BatchProcess_lastJobFinshedAt, _BatchProcess_lastJobFailed, _BatchProcess_starting, _BatchProcess_exited, _BatchProcess_whyNotHealthy, _BatchProcess_taskCount, _BatchProcess_currentTask, _BatchProcess_currentTaskTimeout, _BatchProcess_endPromise, _BatchProcess_execTask, _BatchProcess_end, _BatchProcess_awaitNotRunning, _BatchProcess_onTimeout, _BatchProcess_onError, _BatchProcess_onStderr, _BatchProcess_onStdout, _BatchProcess_clearCurrentTask;
Object.defineProperty(exports, "__esModule", { value: true });
exports.BatchProcess = void 0;
const node_timers_1 = __importDefault(require("node:timers"));
const Async_1 = require("./Async");
const Deferred_1 = require("./Deferred");
const Error_1 = require("./Error");
const Object_1 = require("./Object");
const Parser_1 = require("./Parser");
const Pids_1 = require("./Pids");
const Stream_1 = require("./Stream");
const String_1 = require("./String");
const Task_1 = require("./Task");
const Timeout_1 = require("./Timeout");
/**
* BatchProcess manages the care and feeding of a single child process.
*/
class BatchProcess {
/**
* @param onIdle to be called when internal state changes (like the current
* task is resolved, or the process exits)
*/
constructor(proc, opts, onIdle) {
_BatchProcess_instances.add(this);
this.proc = proc;
this.opts = opts;
this.onIdle = onIdle;
this.start = Date.now();
_BatchProcess_lastHealthCheck.set(this, Date.now());
_BatchProcess_healthCheckFailures.set(this, 0);
_BatchProcess_logger.set(this, void 0);
_BatchProcess_lastJobFinshedAt.set(this, Date.now());
_BatchProcess_lastJobFailed.set(this, false
// Only set to true when `proc.pid` is no longer in the process table.
);
// Only set to true when `proc.pid` is no longer in the process table.
_BatchProcess_starting.set(this, true);
_BatchProcess_exited.set(this, false
// override for .whyNotHealthy()
);
// override for .whyNotHealthy()
_BatchProcess_whyNotHealthy.set(this, void 0);
this.failedTaskCount = 0;
_BatchProcess_taskCount.set(this, -1
/**
* Should be undefined if this instance is not currently processing a task.
*/
); // don't count the startupTask
/**
* Should be undefined if this instance is not currently processing a task.
*/
_BatchProcess_currentTask.set(this, void 0);
_BatchProcess_currentTaskTimeout.set(this, void 0);
_BatchProcess_endPromise.set(this, void 0);
this.name = "BatchProcess(" + proc.pid + ")";
__classPrivateFieldSet(this, _BatchProcess_logger, opts.logger, "f");
// don't let node count the child processes as a reason to stay alive
this.proc.unref();
if (proc.pid == null) {
throw new Error("BatchProcess.constructor: child process pid is null");
}
this.pid = proc.pid;
this.proc.on("error", (err) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onError).call(this, "proc.error", err));
this.proc.on("close", () => this.end(false, "proc.close"));
this.proc.on("exit", () => this.end(false, "proc.exit"));
this.proc.on("disconnect", () => this.end(false, "proc.disconnect"));
const stdin = this.proc.stdin;
if (stdin == null)
throw new Error("Given proc had no stdin");
stdin.on("error", (err) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onError).call(this, "stdin.error", err));
const stdout = this.proc.stdout;
if (stdout == null)
throw new Error("Given proc had no stdout");
stdout.on("error", (err) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onError).call(this, "stdout.error", err));
stdout.on("data", (d) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onStdout).call(this, d));
(0, Object_1.map)(this.proc.stderr, (stderr) => {
stderr.on("error", (err) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onError).call(this, "stderr.error", err));
stderr.on("data", (err) => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onStderr).call(this, err));
});
const startupTask = new Task_1.Task(opts.versionCommand, Parser_1.SimpleParser);
this.startupTaskId = startupTask.taskId;
if (!this.execTask(startupTask)) {
this.opts.observer.emit("internalError", new Error(this.name + " startup task was not submitted"));
}
// this needs to be at the end of the constructor, to ensure everything is
// set up on `this`
this.opts.observer.emit("childStart", this);
}
get currentTask() {
return __classPrivateFieldGet(this, _BatchProcess_currentTask, "f");
}
get taskCount() {
return __classPrivateFieldGet(this, _BatchProcess_taskCount, "f");
}
get starting() {
return __classPrivateFieldGet(this, _BatchProcess_starting, "f");
}
/**
* @return true if `this.end()` has been requested (which may be due to the
* child process exiting)
*/
get ending() {
return __classPrivateFieldGet(this, _BatchProcess_endPromise, "f") != null;
}
/**
* @return true if `this.end()` has completed running, which includes child
* process cleanup. Note that this may return `true` and the process table may
* still include the child pid. Call {@link BatchProcess#running()} for an authoritative
* (but expensive!) answer.
*/
get ended() {
var _a;
return true === ((_a = __classPrivateFieldGet(this, _BatchProcess_endPromise, "f")) === null || _a === void 0 ? void 0 : _a.settled);
}
/**
* @return true if the child process has exited and is no longer in the
* process table. Note that this may be erroneously false if the process table
* hasn't been checked. Call {@link BatchProcess#running()} for an authoritative (but
* expensive!) answer.
*/
get exited() {
return __classPrivateFieldGet(this, _BatchProcess_exited, "f");
}
/**
* @return a string describing why this process should be recycled, or null if
* the process passes all health checks. Note that this doesn't include if
* we're already busy: see {@link BatchProcess.whyNotReady} if you need to
* know if a process can handle a new task.
*/
get whyNotHealthy() {
var _a, _b;
if (__classPrivateFieldGet(this, _BatchProcess_whyNotHealthy, "f") != null)
return __classPrivateFieldGet(this, _BatchProcess_whyNotHealthy, "f");
if (this.ended) {
return "ended";
}
else if (this.ending) {
return "ending";
}
else if (__classPrivateFieldGet(this, _BatchProcess_healthCheckFailures, "f") > 0) {
return "unhealthy";
}
else if (this.proc.stdin == null || this.proc.stdin.destroyed) {
return "closed";
}
else if (this.opts.maxTasksPerProcess > 0 &&
this.taskCount >= this.opts.maxTasksPerProcess) {
return "worn";
}
else if (this.opts.maxIdleMsPerProcess > 0 &&
this.idleMs > this.opts.maxIdleMsPerProcess) {
return "idle";
}
else if (this.opts.maxFailedTasksPerProcess > 0 &&
this.failedTaskCount >= this.opts.maxFailedTasksPerProcess) {
return "broken";
}
else if (this.opts.maxProcAgeMillis > 0 &&
this.start + this.opts.maxProcAgeMillis < Date.now()) {
return "old";
}
else if ((_b = (this.opts.taskTimeoutMillis > 0 && ((_a = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f")) === null || _a === void 0 ? void 0 : _a.runtimeMs))) !== null && _b !== void 0 ? _b : 0 > this.opts.taskTimeoutMillis) {
return "timeout";
}
else {
return null;
}
}
/**
* @return true if the process doesn't need to be recycled.
*/
get healthy() {
return this.whyNotHealthy == null;
}
/**
* @return true iff no current task. Does not take into consideration if the
* process has ended or should be recycled: see {@link BatchProcess.ready}.
*/
get idle() {
return __classPrivateFieldGet(this, _BatchProcess_currentTask, "f") == null;
}
/**
* @return a string describing why this process cannot currently handle a new
* task, or `undefined` if this process is idle and healthy.
*/
get whyNotReady() {
return !this.idle ? "busy" : this.whyNotHealthy;
}
/**
* @return true iff this process is both healthy and idle, and ready for a
* new task.
*/
get ready() {
return this.whyNotReady == null;
}
get idleMs() {
return this.idle ? Date.now() - __classPrivateFieldGet(this, _BatchProcess_lastJobFinshedAt, "f") : -1;
}
/**
* @return true if the child process is in the process table
*/
running() {
if (__classPrivateFieldGet(this, _BatchProcess_exited, "f"))
return false;
const alive = (0, Pids_1.pidExists)(this.pid);
if (!alive) {
__classPrivateFieldSet(this, _BatchProcess_exited, true, "f");
// once a PID leaves the process table, it's gone for good.
this.end(false, "proc.exit");
}
return alive;
}
notRunning() {
return !this.running();
}
maybeRunHealthcheck() {
const hcc = this.opts.healthCheckCommand;
// if there's no health check command, no-op.
if (hcc == null || (0, String_1.blank)(hcc))
return;
// if the prior health check failed, .ready will be false
if (!this.ready)
return;
if (__classPrivateFieldGet(this, _BatchProcess_lastJobFailed, "f") ||
(this.opts.healthCheckIntervalMillis > 0 &&
Date.now() - __classPrivateFieldGet(this, _BatchProcess_lastHealthCheck, "f") >
this.opts.healthCheckIntervalMillis)) {
__classPrivateFieldSet(this, _BatchProcess_lastHealthCheck, Date.now(), "f");
const t = new Task_1.Task(hcc, Parser_1.SimpleParser);
t.promise
.catch((err) => {
var _a;
this.opts.observer.emit("healthCheckError", err, this);
__classPrivateFieldSet(this, _BatchProcess_healthCheckFailures, (_a = __classPrivateFieldGet(this, _BatchProcess_healthCheckFailures, "f"), _a++, _a), "f");
// BatchCluster will see we're unhealthy and reap us later
})
.finally(() => {
__classPrivateFieldSet(this, _BatchProcess_lastHealthCheck, Date.now(), "f");
});
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_execTask).call(this, t);
return t;
}
return;
}
// This must not be async, or new instances aren't started as busy (until the
// startup task is complete)
execTask(task) {
return this.ready ? __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_execTask).call(this, task) : false;
}
/**
* End this child process.
*
* @param gracefully Wait for any current task to be resolved or rejected
* before shutting down the child process.
* @param reason who called end() (used for logging)
* @return Promise that will be resolved when the process has completed.
* Subsequent calls to end() will ignore the parameters and return the first
* endPromise.
*/
// NOT ASYNC! needs to change state immediately.
end(gracefully = true, reason) {
var _a, _b;
return (__classPrivateFieldSet(this, _BatchProcess_endPromise, (_a = __classPrivateFieldGet(this, _BatchProcess_endPromise, "f")) !== null && _a !== void 0 ? _a : new Deferred_1.Deferred().observe(__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_end).call(this, gracefully, (__classPrivateFieldSet(this, _BatchProcess_whyNotHealthy, (_b = __classPrivateFieldGet(this, _BatchProcess_whyNotHealthy, "f")) !== null && _b !== void 0 ? _b : reason, "f")))), "f")).promise;
}
}
exports.BatchProcess = BatchProcess;
_BatchProcess_lastHealthCheck = new WeakMap(), _BatchProcess_healthCheckFailures = new WeakMap(), _BatchProcess_logger = new WeakMap(), _BatchProcess_lastJobFinshedAt = new WeakMap(), _BatchProcess_lastJobFailed = new WeakMap(), _BatchProcess_starting = new WeakMap(), _BatchProcess_exited = new WeakMap(), _BatchProcess_whyNotHealthy = new WeakMap(), _BatchProcess_taskCount = new WeakMap(), _BatchProcess_currentTask = new WeakMap(), _BatchProcess_currentTaskTimeout = new WeakMap(), _BatchProcess_endPromise = new WeakMap(), _BatchProcess_instances = new WeakSet(), _BatchProcess_execTask = function _BatchProcess_execTask(task) {
var _a;
var _b;
if (this.ending)
return false;
__classPrivateFieldSet(this, _BatchProcess_taskCount, (_b = __classPrivateFieldGet(this, _BatchProcess_taskCount, "f"), _b++, _b), "f");
__classPrivateFieldSet(this, _BatchProcess_currentTask, task, "f");
const cmd = (0, String_1.ensureSuffix)(task.command, "\n");
const isStartupTask = task.taskId === this.startupTaskId;
const taskTimeoutMs = isStartupTask
? this.opts.spawnTimeoutMillis
: this.opts.taskTimeoutMillis;
if (taskTimeoutMs > 0) {
// add the stream flush millis to the taskTimeoutMs, because that time
// should not be counted against the task.
__classPrivateFieldSet(this, _BatchProcess_currentTaskTimeout, node_timers_1.default.setTimeout(() => __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onTimeout).call(this, task, taskTimeoutMs), taskTimeoutMs + this.opts.streamFlushMillis), "f");
}
// CAREFUL! If you add a .catch or .finally, the pipeline can emit unhandled
// rejections:
void task.promise.then(() => {
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_clearCurrentTask).call(this, task);
// this.#logger().trace("task completed", { task })
if (isStartupTask) {
// no need to emit taskResolved for startup tasks.
__classPrivateFieldSet(this, _BatchProcess_starting, false, "f");
}
else {
this.opts.observer.emit("taskResolved", task, this);
}
// Call _after_ we've cleared the current task:
this.onIdle();
}, (error) => {
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_clearCurrentTask).call(this, task);
// this.#logger().trace("task failed", { task, err: error })
if (isStartupTask) {
this.opts.observer.emit("startError", error);
this.end(false, "startError");
}
else {
this.opts.observer.emit("taskError", error, task, this);
}
// Call _after_ we've cleared the current task:
this.onIdle();
});
try {
task.onStart(this.opts);
const stdin = (_a = this.proc) === null || _a === void 0 ? void 0 : _a.stdin;
if (stdin == null || stdin.destroyed) {
task.reject(new Error("proc.stdin unexpectedly closed"));
return false;
}
else {
stdin.write(cmd, (err) => {
if (err != null) {
task.reject(err);
}
});
return true;
}
}
catch (err) {
// child process went away. We should too.
this.end(false, "stdin.error");
return false;
}
}, _BatchProcess_end =
// NOTE: Must only be invoked by this.end(), and only expected to be invoked
// once per instance.
async function _BatchProcess_end(gracefully, reason) {
var _a, _b, _c, _d;
const lastTask = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f");
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_clearCurrentTask).call(this);
// NOTE: We wait on all tasks (even startup tasks) so we can assert that
// BatchCluster is idle (and this proc is idle) when the end promise is
// resolved.
// NOTE: holy crap there are a lot of notes here.
// We don't need to wait for the startup task to complete, and we certainly
// don't need to fuss about ending when we're just getting started.
if (lastTask != null && lastTask.taskId !== this.startupTaskId) {
try {
// Let's wait for the process to complete and the streams to flush, as
// that may actually allow the task to complete successfully. Let's not
// wait forever, though.
await (0, Timeout_1.thenOrTimeout)(lastTask.promise, gracefully ? 2000 : 250);
}
catch {
//
}
if (lastTask.pending) {
lastTask.reject(new Error(`end() called before task completed (${JSON.stringify({
gracefully,
lastTask,
})})`));
}
}
// Ignore EPIPE on .end(): if the process immediately ends after the exit
// command, we'll get an EPIPE, so, shush error events *before* we tell the
// child process to exit. See https://github.com/nodejs/node/issues/26828
for (const ea of [
this.proc,
this.proc.stdin,
this.proc.stdout,
this.proc.stderr,
]) {
ea === null || ea === void 0 ? void 0 : ea.removeAllListeners("error");
}
if (true === ((_a = this.proc.stdin) === null || _a === void 0 ? void 0 : _a.writable)) {
const exitCmd = this.opts.exitCommand == null
? null
: (0, String_1.ensureSuffix)(this.opts.exitCommand, "\n");
try {
(_b = this.proc.stdin) === null || _b === void 0 ? void 0 : _b.end(exitCmd);
}
catch {
// don't care
}
}
// None of this *should* be necessary, but we're trying to be as hygienic as
// we can to avoid process zombification.
(0, Stream_1.destroy)(this.proc.stdin);
(0, Stream_1.destroy)(this.proc.stdout);
(0, Stream_1.destroy)(this.proc.stderr);
if (this.opts.cleanupChildProcs &&
gracefully &&
this.opts.endGracefulWaitTimeMillis > 0 &&
!__classPrivateFieldGet(this, _BatchProcess_exited, "f")) {
// Wait for the exit command to take effect:
await __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_awaitNotRunning).call(this, this.opts.endGracefulWaitTimeMillis / 2);
// If it's still running, send the pid a signal:
if (this.running() && this.proc.pid != null)
this.proc.kill();
// Wait for the signal handler to work:
await __classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_awaitNotRunning).call(this, this.opts.endGracefulWaitTimeMillis / 2);
}
if (this.opts.cleanupChildProcs &&
this.proc.pid != null &&
this.running()) {
__classPrivateFieldGet(this, _BatchProcess_logger, "f").call(this).warn(this.name + ".end(): force-killing still-running child.");
(0, Pids_1.kill)(this.proc.pid, true);
}
// disconnect may not be a function on proc!
(_d = (_c = this.proc).disconnect) === null || _d === void 0 ? void 0 : _d.call(_c);
this.opts.observer.emit("childEnd", this, reason);
}, _BatchProcess_awaitNotRunning = function _BatchProcess_awaitNotRunning(timeout) {
return (0, Async_1.until)(() => this.notRunning(), timeout);
}, _BatchProcess_onTimeout = function _BatchProcess_onTimeout(task, timeoutMs) {
if (task.pending) {
this.opts.observer.emit("taskTimeout", timeoutMs, task, this);
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_onError).call(this, "timeout", new Error("waited " + timeoutMs + "ms"), task);
}
}, _BatchProcess_onError = function _BatchProcess_onError(reason, error, task) {
if (task == null) {
task = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f");
}
const cleanedError = new Error(reason + ": " + (0, Error_1.cleanError)(error.message));
if (error.stack != null) {
// Error stacks, if set, will not be redefined from a rethrow:
cleanedError.stack = (0, Error_1.cleanError)(error.stack);
}
__classPrivateFieldGet(this, _BatchProcess_logger, "f").call(this).warn(this.name + ".onError()", {
reason,
task: (0, Object_1.map)(task, (t) => t.command),
error: cleanedError,
});
if (this.ending) {
// .#end is already disconnecting the error listeners, but in any event,
// we don't really care about errors after we've been told to shut down.
return;
}
// clear the task before ending so the onExit from end() doesn't retry the task:
__classPrivateFieldGet(this, _BatchProcess_instances, "m", _BatchProcess_clearCurrentTask).call(this);
void this.end(false, reason);
if (task != null && this.taskCount === 1) {
__classPrivateFieldGet(this, _BatchProcess_logger, "f").call(this).warn(this.name + ".onError(): startup task failed: " + cleanedError);
this.opts.observer.emit("startError", cleanedError);
}
if (task != null) {
if (task.pending) {
task.reject(cleanedError);
}
else {
this.opts.observer.emit("internalError", new Error(`${this.name}.onError(${cleanedError}) cannot reject already-fulfilled task.`));
}
}
}, _BatchProcess_onStderr = function _BatchProcess_onStderr(data) {
if ((0, String_1.blank)(data))
return;
__classPrivateFieldGet(this, _BatchProcess_logger, "f").call(this).warn(this.name + ".onStderr(): " + data);
const task = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f");
if (task != null && task.pending) {
task.onStderr(data);
}
else if (!this.ending) {
// If we're ending and there isn't a task, don't worry about it.
this.opts.observer.emit("noTaskData", null, data, this);
void this.end(false, "stderr");
}
}, _BatchProcess_onStdout = function _BatchProcess_onStdout(data) {
if (data == null)
return;
const task = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f");
if (task != null && task.pending) {
this.opts.observer.emit("taskData", data, task, this);
task.onStdout(data);
}
else if (this.ending) {
// don't care if we're already being shut down.
}
else if (!(0, String_1.blank)(data)) {
this.opts.observer.emit("noTaskData", data, null, this);
void this.end(false, "stdout.error");
}
}, _BatchProcess_clearCurrentTask = function _BatchProcess_clearCurrentTask(task) {
var _a;
__classPrivateFieldSet(this, _BatchProcess_lastJobFailed, (task === null || task === void 0 ? void 0 : task.state) === "rejected", "f");
if (task != null && task.taskId !== ((_a = __classPrivateFieldGet(this, _BatchProcess_currentTask, "f")) === null || _a === void 0 ? void 0 : _a.taskId))
return;
(0, Object_1.map)(__classPrivateFieldGet(this, _BatchProcess_currentTaskTimeout, "f"), (ea) => clearTimeout(ea));
__classPrivateFieldSet(this, _BatchProcess_currentTaskTimeout, undefined, "f");
__classPrivateFieldSet(this, _BatchProcess_currentTask, undefined, "f");
__classPrivateFieldSet(this, _BatchProcess_lastJobFinshedAt, Date.now(), "f");
};
//# sourceMappingURL=BatchProcess.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"BatchProcess.js","sourceRoot":"","sources":["../src/BatchProcess.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;AACA,8DAAgC;AAChC,mCAA+B;AAC/B,yCAAqC;AACrC,mCAAoC;AAGpC,qCAA8B;AAC9B,qCAAuC;AACvC,iCAAwC;AACxC,qCAAkC;AAClC,qCAA8C;AAC9C,iCAA6B;AAC7B,uCAAyC;AAyBzC;;GAEG;AACH,MAAa,YAAY;IAgCvB;;;OAGG;IACH,YACW,IAAgC,EAChC,IAAiC,EACzB,MAAkB;;QAF1B,SAAI,GAAJ,IAAI,CAA4B;QAChC,SAAI,GAAJ,IAAI,CAA6B;QACzB,WAAM,GAAN,MAAM,CAAY;QApC5B,UAAK,GAAG,IAAI,CAAC,GAAG,EAAE,CAAA;QAC3B,wCAAmB,IAAI,CAAC,GAAG,EAAE,EAAA;QAC7B,4CAAuB,CAAC,EAAA;QAGf,uCAAqB;QAC9B,yCAAoB,IAAI,CAAC,GAAG,EAAE,EAAA;QAC9B,sCAAiB,KAAK;QAEtB,sEAAsE;UAFhD;QAEtB,sEAAsE;QACtE,iCAAY,IAAI,EAAA;QAEhB,+BAAU,KAAK;QAEf,gCAAgC;UAFjB;QAEf,gCAAgC;QAChC,8CAA8B;QAE9B,oBAAe,GAAG,CAAC,CAAA;QAEnB,kCAAa,CAAC,CAAC;QAEf;;WAEG;UAJY,CAAC,8BAA8B;QAE9C;;WAEG;QACH,4CAA8B;QAC9B,mDAA+C;QAE/C,2CAAuC;QAWrC,IAAI,CAAC,IAAI,GAAG,eAAe,GAAG,IAAI,CAAC,GAAG,GAAG,GAAG,CAAA;QAC5C,uBAAA,IAAI,wBAAW,IAAI,CAAC,MAAM,MAAA,CAAA;QAC1B,qEAAqE;QACrE,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAA;QAEjB,IAAI,IAAI,CAAC,GAAG,IAAI,IAAI,EAAE,CAAC;YACrB,MAAM,IAAI,KAAK,CAAC,qDAAqD,CAAC,CAAA;QACxE,CAAC;QAED,IAAI,CAAC,GAAG,GAAG,IAAI,CAAC,GAAG,CAAA;QAEnB,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,uBAAA,IAAI,sDAAS,MAAb,IAAI,EAAU,YAAY,EAAE,GAAG,CAAC,CAAC,CAAA;QAChE,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,OAAO,EAAE,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,YAAY,CAAC,CAAC,CAAA;QAC1D,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,MAAM,EAAE,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,WAAW,CAAC,CAAC,CAAA;QACxD,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,YAAY,EAAE,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,iBAAiB,CAAC,CAAC,CAAA;QAEpE,MAAM,KAAK,GAAG,IAAI,CAAC,IAAI,CAAC,KAAK,CAAA;QAC7B,IAAI,KAAK,IAAI,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,yBAAyB,CAAC,CAAA;QAC7D,KAAK,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,uBAAA,IAAI,sDAAS,MAAb,IAAI,EAAU,aAAa,EAAE,GAAG,CAAC,CAAC,CAAA;QAE7D,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,MAAM,CAAA;QAC/B,IAAI,MAAM,IAAI,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,0BAA0B,CAAC,CAAA;QAC/D,MAAM,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,uBAAA,IAAI,sDAAS,MAAb,IAAI,EAAU,cAAc,EAAE,GAAG,CAAC,CAAC,CAAA;QAC/D,MAAM,CAAC,EAAE,CAAC,MAAM,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,uBAAA,IAAI,uDAAU,MAAd,IAAI,EAAW,CAAC,CAAC,CAAC,CAAA;QAE3C,IAAA,YAAG,EAAC,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,MAAM,EAAE,EAAE;YAC/B,MAAM,CAAC,EAAE,CAAC,OAAO,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,uBAAA,IAAI,sDAAS,MAAb,IAAI,EAAU,cAAc,EAAE,GAAG,CAAC,CAAC,CAAA;YAC/D,MAAM,CAAC,EAAE,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,uBAAA,IAAI,uDAAU,MAAd,IAAI,EAAW,GAAG,CAAC,CAAC,CAAA;QACjD,CAAC,CAAC,CAAA;QAEF,MAAM,WAAW,GAAG,IAAI,WAAI,CAAC,IAAI,CAAC,cAAc,EAAE,qBAAY,CAAC,CAAA;QAC/D,IAAI,CAAC,aAAa,GAAG,WAAW,CAAC,MAAM,CAAA;QAEvC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,WAAW,CAAC,EAAE,CAAC;YAChC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,CACrB,eAAe,EACf,IAAI,KAAK,CAAC,IAAI,CAAC,IAAI,GAAG,iCAAiC,CAAC,CACzD,CAAA;QACH,CAAC;QACD,0EAA0E;QAC1E,mBAAmB;QACnB,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,YAAY,EAAE,IAAI,CAAC,CAAA;IAC7C,CAAC;IAED,IAAI,WAAW;QACb,OAAO,uBAAA,IAAI,iCAAa,CAAA;IAC1B,CAAC;IAED,IAAI,SAAS;QACX,OAAO,uBAAA,IAAI,+BAAW,CAAA;IACxB,CAAC;IAED,IAAI,QAAQ;QACV,OAAO,uBAAA,IAAI,8BAAU,CAAA;IACvB,CAAC;IAED;;;OAGG;IACH,IAAI,MAAM;QACR,OAAO,uBAAA,IAAI,gCAAY,IAAI,IAAI,CAAA;IACjC,CAAC;IAED;;;;;OAKG;IACH,IAAI,KAAK;;QACP,OAAO,IAAI,MAAK,MAAA,uBAAA,IAAI,gCAAY,0CAAE,OAAO,CAAA,CAAA;IAC3C,CAAC;IAED;;;;;OAKG;IACH,IAAI,MAAM;QACR,OAAO,uBAAA,IAAI,4BAAQ,CAAA;IACrB,CAAC;IAED;;;;;OAKG;IACH,IAAI,aAAa;;QACf,IAAI,uBAAA,IAAI,mCAAe,IAAI,IAAI;YAAE,OAAO,uBAAA,IAAI,mCAAe,CAAA;QAC3D,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;YACf,OAAO,OAAO,CAAA;QAChB,CAAC;aAAM,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;YACvB,OAAO,QAAQ,CAAA;QACjB,CAAC;aAAM,IAAI,uBAAA,IAAI,yCAAqB,GAAG,CAAC,EAAE,CAAC;YACzC,OAAO,WAAW,CAAA;QACpB,CAAC;aAAM,IAAI,IAAI,CAAC,IAAI,CAAC,KAAK,IAAI,IAAI,IAAI,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,SAAS,EAAE,CAAC;YAChE,OAAO,QAAQ,CAAA;QACjB,CAAC;aAAM,IACL,IAAI,CAAC,IAAI,CAAC,kBAAkB,GAAG,CAAC;YAChC,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,IAAI,CAAC,kBAAkB,EAC9C,CAAC;YACD,OAAO,MAAM,CAAA;QACf,CAAC;aAAM,IACL,IAAI,CAAC,IAAI,CAAC,mBAAmB,GAAG,CAAC;YACjC,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC3C,CAAC;YACD,OAAO,MAAM,CAAA;QACf,CAAC;aAAM,IACL,IAAI,CAAC,IAAI,CAAC,wBAAwB,GAAG,CAAC;YACtC,IAAI,CAAC,eAAe,IAAI,IAAI,CAAC,IAAI,CAAC,wBAAwB,EAC1D,CAAC;YACD,OAAO,QAAQ,CAAA;QACjB,CAAC;aAAM,IACL,IAAI,CAAC,IAAI,CAAC,gBAAgB,GAAG,CAAC;YAC9B,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,IAAI,CAAC,gBAAgB,GAAG,IAAI,CAAC,GAAG,EAAE,EACpD,CAAC;YACD,OAAO,KAAK,CAAA;QACd,CAAC;aAAM,IACL,MAAA,CAAC,IAAI,CAAC,IAAI,CAAC,iBAAiB,GAAG,CAAC,KAAI,MAAA,uBAAA,IAAI,iCAAa,0CAAE,SAAS,CAAA,CAAC,mCACjE,CAAC,GAAG,IAAI,CAAC,IAAI,CAAC,iBAAiB,EAC/B,CAAC;YACD,OAAO,SAAS,CAAA;QAClB,CAAC;aAAM,CAAC;YACN,OAAO,IAAI,CAAA;QACb,CAAC;IACH,CAAC;IAED;;OAEG;IACH,IAAI,OAAO;QACT,OAAO,IAAI,CAAC,aAAa,IAAI,IAAI,CAAA;IACnC,CAAC;IAED;;;OAGG;IACH,IAAI,IAAI;QACN,OAAO,uBAAA,IAAI,iCAAa,IAAI,IAAI,CAAA;IAClC,CAAC;IAED;;;OAGG;IACH,IAAI,WAAW;QACb,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAA;IACjD,CAAC;IAED;;;OAGG;IACH,IAAI,KAAK;QACP,OAAO,IAAI,CAAC,WAAW,IAAI,ILine truncated
+39
View File
@@ -0,0 +1,39 @@
/**
* `BatchProcessOptions` have no reasonable defaults, as they are specific to
* the API of the command that BatchCluster is spawning.
*
* All fields must be set.
*/
export interface BatchProcessOptions {
/**
* Low-overhead command to verify the child batch process has started. Will
* be invoked immediately after spawn. This command must return before any
* tasks will be given to a given process.
*/
versionCommand: string;
/**
* If provided, and healthCheckIntervalMillis is greater than 0, or the
* previous task failed, this command will be sent to child processes.
*
* If the command outputs to stderr or returns a fail string, the process will
* be considered unhealthy and recycled.
*/
healthCheckCommand?: string | undefined;
/**
* Expected text to print if a command passes. Cannot be blank. Strings will
* be interpreted as a regular expression fragment.
*/
pass: string | RegExp;
/**
* Expected text to print if a command fails. Cannot be blank. Strings will
* be interpreted as a regular expression fragment.
*/
fail: string | RegExp;
/**
* Command to end the child batch process. If not provided (or undefined),
* stdin will be closed to signal to the child process that it may terminate,
* and if it does not shut down within `endGracefulWaitTimeMillis`, it will be
* SIGHUP'ed.
*/
exitCommand?: string | undefined;
}
+3
View File
@@ -0,0 +1,3 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
//# sourceMappingURL=BatchProcessOptions.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"BatchProcessOptions.js","sourceRoot":"","sources":["../src/BatchProcessOptions.ts"],"names":[],"mappings":""}
+33
View File
@@ -0,0 +1,33 @@
/**
* Enables a Promise to be resolved or rejected at a future time, outside of
* the context of the Promise construction. Also exposes the `pending`,
* `fulfilled`, or `rejected` state of the promise.
*/
export declare class Deferred<T> implements PromiseLike<T> {
#private;
readonly [Symbol.toStringTag] = "Deferred";
readonly promise: Promise<T>;
constructor();
/**
* @return `true` iff neither `resolve` nor `rejected` have been invoked
*/
get pending(): boolean;
/**
* @return `true` iff `resolve` has been invoked
*/
get fulfilled(): boolean;
/**
* @return `true` iff `rejected` has been invoked
*/
get rejected(): boolean;
/**
* @return `true` iff `resolve` or `rejected` have been invoked
*/
get settled(): boolean;
then<TResult1 = T, TResult2 = never>(onfulfilled?: ((value: T) => TResult1 | PromiseLike<TResult1>) | undefined | null, onrejected?: ((reason: any) => TResult2 | PromiseLike<TResult2>) | undefined | null): Promise<TResult1 | TResult2>;
catch<TResult = never>(onrejected?: ((reason: any) => TResult | PromiseLike<TResult>) | undefined | null): Promise<T | TResult>;
resolve(value: T): boolean;
reject(reason?: Error | string): boolean;
observe(p: Promise<T>): this;
observeQuietly(p: Promise<T>): Deferred<T | undefined>;
}
+122
View File
@@ -0,0 +1,122 @@
"use strict";
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _Deferred_resolve, _Deferred_reject, _Deferred_state, _a;
Object.defineProperty(exports, "__esModule", { value: true });
exports.Deferred = void 0;
/**
* @see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
*/
var State;
(function (State) {
State[State["pending"] = 0] = "pending";
State[State["fulfilled"] = 1] = "fulfilled";
State[State["rejected"] = 2] = "rejected";
})(State || (State = {}));
/**
* Enables a Promise to be resolved or rejected at a future time, outside of
* the context of the Promise construction. Also exposes the `pending`,
* `fulfilled`, or `rejected` state of the promise.
*/
class Deferred {
constructor() {
this[_a] = "Deferred";
_Deferred_resolve.set(this, void 0);
_Deferred_reject.set(this, void 0);
_Deferred_state.set(this, State.pending);
this.promise = new Promise((resolve, reject) => {
__classPrivateFieldSet(this, _Deferred_resolve, resolve, "f");
__classPrivateFieldSet(this, _Deferred_reject, reject, "f");
});
}
/**
* @return `true` iff neither `resolve` nor `rejected` have been invoked
*/
get pending() {
return __classPrivateFieldGet(this, _Deferred_state, "f") === State.pending;
}
/**
* @return `true` iff `resolve` has been invoked
*/
get fulfilled() {
return __classPrivateFieldGet(this, _Deferred_state, "f") === State.fulfilled;
}
/**
* @return `true` iff `rejected` has been invoked
*/
get rejected() {
return __classPrivateFieldGet(this, _Deferred_state, "f") === State.rejected;
}
/**
* @return `true` iff `resolve` or `rejected` have been invoked
*/
get settled() {
return __classPrivateFieldGet(this, _Deferred_state, "f") !== State.pending;
}
then(onfulfilled, onrejected) {
return this.promise.then(onfulfilled, onrejected);
}
catch(onrejected) {
return this.promise.catch(onrejected);
}
resolve(value) {
if (this.settled) {
return false;
}
else {
__classPrivateFieldSet(this, _Deferred_state, State.fulfilled, "f");
__classPrivateFieldGet(this, _Deferred_resolve, "f").call(this, value);
return true;
}
}
reject(reason) {
const wasSettled = this.settled;
// This isn't great: the wrapped Promise may be in a different state than
// #state: but the caller wanted to reject, so even if it already was
// resolved, let's try to respect that.
__classPrivateFieldSet(this, _Deferred_state, State.rejected, "f");
if (wasSettled) {
return false;
}
else {
__classPrivateFieldGet(this, _Deferred_reject, "f").call(this, reason);
return true;
}
}
observe(p) {
void observe(this, p);
return this;
}
observeQuietly(p) {
void observeQuietly(this, p);
return this;
}
}
exports.Deferred = Deferred;
_Deferred_resolve = new WeakMap(), _Deferred_reject = new WeakMap(), _Deferred_state = new WeakMap(), _a = Symbol.toStringTag;
async function observe(d, p) {
try {
d.resolve(await p);
}
catch (err) {
d.reject(err);
}
}
async function observeQuietly(d, p) {
try {
d.resolve(await p);
}
catch {
d.resolve(undefined);
}
}
//# sourceMappingURL=Deferred.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Deferred.js","sourceRoot":"","sources":["../src/Deferred.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;AAAA;;GAEG;AACH,IAAK,KAIJ;AAJD,WAAK,KAAK;IACR,uCAAO,CAAA;IACP,2CAAS,CAAA;IACT,yCAAQ,CAAA;AACV,CAAC,EAJI,KAAK,KAAL,KAAK,QAIT;AAED;;;;GAIG;AACH,MAAa,QAAQ;IAOnB;QANS,QAAoB,GAAG,UAAU,CAAA;QAE1C,oCAA8C;QAC9C,mCAAgC;QAChC,0BAAgB,KAAK,CAAC,OAAO,EAAA;QAG3B,IAAI,CAAC,OAAO,GAAG,IAAI,OAAO,CAAI,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;YAChD,uBAAA,IAAI,qBAAY,OAAO,MAAA,CAAA;YACvB,uBAAA,IAAI,oBAAW,MAAM,MAAA,CAAA;QACvB,CAAC,CAAC,CAAA;IACJ,CAAC;IAED;;OAEG;IACH,IAAI,OAAO;QACT,OAAO,uBAAA,IAAI,uBAAO,KAAK,KAAK,CAAC,OAAO,CAAA;IACtC,CAAC;IAED;;OAEG;IACH,IAAI,SAAS;QACX,OAAO,uBAAA,IAAI,uBAAO,KAAK,KAAK,CAAC,SAAS,CAAA;IACxC,CAAC;IAED;;OAEG;IACH,IAAI,QAAQ;QACV,OAAO,uBAAA,IAAI,uBAAO,KAAK,KAAK,CAAC,QAAQ,CAAA;IACvC,CAAC;IAED;;OAEG;IACH,IAAI,OAAO;QACT,OAAO,uBAAA,IAAI,uBAAO,KAAK,KAAK,CAAC,OAAO,CAAA;IACtC,CAAC;IAED,IAAI,CACF,WAGQ,EACR,UAGQ;QAER,OAAO,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,WAAW,EAAE,UAAU,CAAC,CAAA;IACnD,CAAC;IAED,KAAK,CACH,UAGQ;QAER,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,UAAU,CAAC,CAAA;IACvC,CAAC;IAED,OAAO,CAAC,KAAQ;QACd,IAAI,IAAI,CAAC,OAAO,EAAE,CAAC;YACjB,OAAO,KAAK,CAAA;QACd,CAAC;aAAM,CAAC;YACN,uBAAA,IAAI,mBAAU,KAAK,CAAC,SAAS,MAAA,CAAA;YAC7B,uBAAA,IAAI,yBAAS,MAAb,IAAI,EAAU,KAAK,CAAC,CAAA;YACpB,OAAO,IAAI,CAAA;QACb,CAAC;IACH,CAAC;IAED,MAAM,CAAC,MAAuB;QAC5B,MAAM,UAAU,GAAG,IAAI,CAAC,OAAO,CAAA;QAC/B,yEAAyE;QACzE,qEAAqE;QACrE,uCAAuC;QACvC,uBAAA,IAAI,mBAAU,KAAK,CAAC,QAAQ,MAAA,CAAA;QAC5B,IAAI,UAAU,EAAE,CAAC;YACf,OAAO,KAAK,CAAA;QACd,CAAC;aAAM,CAAC;YACN,uBAAA,IAAI,wBAAQ,MAAZ,IAAI,EAAS,MAAM,CAAC,CAAA;YACpB,OAAO,IAAI,CAAA;QACb,CAAC;IACH,CAAC;IAED,OAAO,CAAC,CAAa;QACnB,KAAK,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,CAAA;QACrB,OAAO,IAAI,CAAA;IACb,CAAC;IAED,cAAc,CAAC,CAAa;QAC1B,KAAK,cAAc,CAAC,IAAI,EAAE,CAAC,CAAC,CAAA;QAC5B,OAAO,IAAW,CAAA;IACpB,CAAC;CACF;AAjGD,4BAiGC;2GAhGW,MAAM,CAAC,WAAW;AAkG9B,KAAK,UAAU,OAAO,CAAI,CAAc,EAAE,CAAa;IACrD,IAAI,CAAC;QACH,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAA;IACpB,CAAC;IAAC,OAAO,GAAQ,EAAE,CAAC;QAClB,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,CAAA;IACf,CAAC;AACH,CAAC;AAED,KAAK,UAAU,cAAc,CAAI,CAAc,EAAE,CAAa;IAC5D,IAAI,CAAC;QACH,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAA;IACpB,CAAC;IAAC,MAAM,CAAC;QACP,CAAC,CAAC,OAAO,CAAC,SAAgB,CAAC,CAAA;IAC7B,CAAC;AACH,CAAC"}
+7
View File
@@ -0,0 +1,7 @@
/**
* When we wrap errors, an Error always prefixes the toString() and stack with
* "Error: ", so we can remove that prefix.
*/
export declare function tryEach(arr: (() => void)[]): void;
export declare function cleanError(s: any): string;
export declare function asError(err: any): Error;
+32
View File
@@ -0,0 +1,32 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.asError = exports.cleanError = exports.tryEach = void 0;
const String_1 = require("./String");
/**
* When we wrap errors, an Error always prefixes the toString() and stack with
* "Error: ", so we can remove that prefix.
*/
function tryEach(arr) {
for (const f of arr) {
try {
f();
}
catch (_) {
//
}
}
}
exports.tryEach = tryEach;
function cleanError(s) {
return String(s)
.trim()
.replace(/^error: /i, "");
}
exports.cleanError = cleanError;
function asError(err) {
return err instanceof Error
? err
: new Error((0, String_1.blank)(err) ? "(unknown)" : (0, String_1.toS)(err));
}
exports.asError = asError;
//# sourceMappingURL=Error.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Error.js","sourceRoot":"","sources":["../src/Error.ts"],"names":[],"mappings":";;;AAAA,qCAAqC;AAErC;;;GAGG;AACH,SAAgB,OAAO,CAAC,GAAmB;IACzC,KAAK,MAAM,CAAC,IAAI,GAAG,EAAE,CAAC;QACpB,IAAI,CAAC;YACH,CAAC,EAAE,CAAA;QACL,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACX,EAAE;QACJ,CAAC;IACH,CAAC;AACH,CAAC;AARD,0BAQC;AAED,SAAgB,UAAU,CAAC,CAAM;IAC/B,OAAO,MAAM,CAAC,CAAC,CAAC;SACb,IAAI,EAAE;SACN,OAAO,CAAC,WAAW,EAAE,EAAE,CAAC,CAAA;AAC7B,CAAC;AAJD,gCAIC;AAED,SAAgB,OAAO,CAAC,GAAQ;IAC9B,OAAO,GAAG,YAAY,KAAK;QACzB,CAAC,CAAC,GAAG;QACL,CAAC,CAAC,IAAI,KAAK,CAAC,IAAA,cAAK,EAAC,GAAG,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,IAAA,YAAG,EAAC,GAAG,CAAC,CAAC,CAAA;AACpD,CAAC;AAJD,0BAIC"}
+6
View File
@@ -0,0 +1,6 @@
import { BatchClusterOptions, WithObserver } from "./BatchClusterOptions";
import { BatchProcessOptions } from "./BatchProcessOptions";
export interface InternalBatchProcessOptions extends BatchProcessOptions, BatchClusterOptions, WithObserver {
passRE: RegExp;
failRE: RegExp;
}
+3
View File
@@ -0,0 +1,3 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
//# sourceMappingURL=InternalBatchProcessOptions.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"InternalBatchProcessOptions.js","sourceRoot":"","sources":["../src/InternalBatchProcessOptions.ts"],"names":[],"mappings":""}
+35
View File
@@ -0,0 +1,35 @@
type LogFunc = (message: string, ...optionalParams: any[]) => void;
/**
* Simple interface for logging.
*/
export interface Logger {
trace: LogFunc;
debug: LogFunc;
info: LogFunc;
warn: LogFunc;
error: LogFunc;
}
export declare const LogLevels: (keyof Logger)[];
/**
* Default `Logger` implementation.
*
* - `debug` and `info` go to {@link util.debuglog}("batch-cluster")`.
*
* - `warn` and `error` go to `console.warn` and `console.error`.
*
* @see https://nodejs.org/api/util.html#util_util_debuglog_section
* @see https://nodejs.org/api/console.html
*/
export declare const ConsoleLogger: Logger;
/**
* `Logger` that disables all logging.
*/
export declare const NoLogger: Logger;
export declare function setLogger(l: Logger): void;
export declare function logger(): Logger;
export declare const Log: {
withLevels: (delegate: Logger) => Logger;
withTimestamps: (delegate: Logger) => any;
filterLevels: (l: Logger, minLogLevel: keyof Logger) => any;
};
export {};
+100
View File
@@ -0,0 +1,100 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.Log = exports.logger = exports.setLogger = exports.NoLogger = exports.ConsoleLogger = exports.LogLevels = void 0;
const node_util_1 = __importDefault(require("node:util"));
const Object_1 = require("./Object");
const String_1 = require("./String");
exports.LogLevels = [
"trace",
"debug",
"info",
"warn",
"error",
];
const _debuglog = node_util_1.default.debuglog("batch-cluster");
const noop = () => undefined;
/**
* Default `Logger` implementation.
*
* - `debug` and `info` go to {@link util.debuglog}("batch-cluster")`.
*
* - `warn` and `error` go to `console.warn` and `console.error`.
*
* @see https://nodejs.org/api/util.html#util_util_debuglog_section
* @see https://nodejs.org/api/console.html
*/
exports.ConsoleLogger = Object.freeze({
/**
* No-ops by default, as this is very low-level information.
*/
trace: noop,
/**
* Delegates to `util.debuglog("batch-cluster")`:
* <https://nodejs.org/api/util.html#util_util_debuglog_section>
*/
debug: _debuglog,
/**
* Delegates to `util.debuglog("batch-cluster")`:
* <https://nodejs.org/api/util.html#util_util_debuglog_section>
*/
info: _debuglog,
/**
* Delegates to `console.warn`
*/
warn: console.warn,
/**
* Delegates to `console.error`
*/
error: console.error,
});
/**
* `Logger` that disables all logging.
*/
exports.NoLogger = Object.freeze({
trace: noop,
debug: noop,
info: noop,
warn: noop,
error: noop,
});
let _logger = exports.NoLogger;
function setLogger(l) {
if (exports.LogLevels.some((ea) => typeof l[ea] !== "function")) {
throw new Error("invalid logger, must implement " + exports.LogLevels);
}
_logger = l;
}
exports.setLogger = setLogger;
function logger() {
return _logger;
}
exports.logger = logger;
exports.Log = {
withLevels: (delegate) => {
const timestamped = {};
exports.LogLevels.forEach((ea) => {
const prefix = (ea + " ").substring(0, 5) + " | ";
timestamped[ea] = (message, ...optionalParams) => {
if ((0, String_1.notBlank)(message)) {
delegate[ea](prefix + message, ...optionalParams);
}
};
});
return timestamped;
},
withTimestamps: (delegate) => {
const timestamped = {};
exports.LogLevels.forEach((level) => (timestamped[level] = (message, ...optionalParams) => (0, Object_1.map)(message, (ea) => delegate[level](new Date().toISOString() + " | " + ea, ...optionalParams))));
return timestamped;
},
filterLevels: (l, minLogLevel) => {
const minLogLevelIndex = exports.LogLevels.indexOf(minLogLevel);
const filtered = {};
exports.LogLevels.forEach((ea, idx) => (filtered[ea] = idx < minLogLevelIndex ? noop : l[ea].bind(l)));
return filtered;
},
};
//# sourceMappingURL=Logger.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Logger.js","sourceRoot":"","sources":["../src/Logger.ts"],"names":[],"mappings":";;;;;;AAAA,0DAA4B;AAC5B,qCAA8B;AAC9B,qCAAmC;AAetB,QAAA,SAAS,GAAqB;IACzC,OAAO;IACP,OAAO;IACP,MAAM;IACN,MAAM;IACN,OAAO;CACR,CAAA;AAED,MAAM,SAAS,GAAG,mBAAI,CAAC,QAAQ,CAAC,eAAe,CAAC,CAAA;AAEhD,MAAM,IAAI,GAAG,GAAG,EAAE,CAAC,SAAS,CAAA;AAE5B;;;;;;;;;GASG;AACU,QAAA,aAAa,GAAW,MAAM,CAAC,MAAM,CAAC;IACjD;;OAEG;IACH,KAAK,EAAE,IAAI;IAEX;;;OAGG;IACH,KAAK,EAAE,SAAS;IAChB;;;OAGG;IACH,IAAI,EAAE,SAAS;IACf;;OAEG;IACH,IAAI,EAAE,OAAO,CAAC,IAAI;IAClB;;OAEG;IACH,KAAK,EAAE,OAAO,CAAC,KAAK;CACrB,CAAC,CAAA;AAEF;;GAEG;AACU,QAAA,QAAQ,GAAW,MAAM,CAAC,MAAM,CAAC;IAC5C,KAAK,EAAE,IAAI;IACX,KAAK,EAAE,IAAI;IACX,IAAI,EAAE,IAAI;IACV,IAAI,EAAE,IAAI;IACV,KAAK,EAAE,IAAI;CACZ,CAAC,CAAA;AAEF,IAAI,OAAO,GAAW,gBAAQ,CAAA;AAE9B,SAAgB,SAAS,CAAC,CAAS;IACjC,IAAI,iBAAS,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,KAAK,UAAU,CAAC,EAAE,CAAC;QACxD,MAAM,IAAI,KAAK,CAAC,iCAAiC,GAAG,iBAAS,CAAC,CAAA;IAChE,CAAC;IACD,OAAO,GAAG,CAAC,CAAA;AACb,CAAC;AALD,8BAKC;AAED,SAAgB,MAAM;IACpB,OAAO,OAAO,CAAA;AAChB,CAAC;AAFD,wBAEC;AAEY,QAAA,GAAG,GAAG;IACjB,UAAU,EAAE,CAAC,QAAgB,EAAU,EAAE;QACvC,MAAM,WAAW,GAAQ,EAAE,CAAA;QAC3B,iBAAS,CAAC,OAAO,CAAC,CAAC,EAAE,EAAE,EAAE;YACvB,MAAM,MAAM,GAAG,CAAC,EAAE,GAAG,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,KAAK,CAAA;YACjD,WAAW,CAAC,EAAE,CAAC,GAAG,CAAC,OAAa,EAAE,GAAG,cAAqB,EAAE,EAAE;gBAC5D,IAAI,IAAA,iBAAQ,EAAC,OAAO,CAAC,EAAE,CAAC;oBACtB,QAAQ,CAAC,EAAE,CAAC,CAAC,MAAM,GAAG,OAAO,EAAE,GAAG,cAAc,CAAC,CAAA;gBACnD,CAAC;YACH,CAAC,CAAA;QACH,CAAC,CAAC,CAAA;QACF,OAAO,WAAW,CAAA;IACpB,CAAC;IAED,cAAc,EAAE,CAAC,QAAgB,EAAE,EAAE;QACnC,MAAM,WAAW,GAAQ,EAAE,CAAA;QAC3B,iBAAS,CAAC,OAAO,CACf,CAAC,KAAK,EAAE,EAAE,CACR,CAAC,WAAW,CAAC,KAAK,CAAC,GAAG,CAAC,OAAa,EAAE,GAAG,cAAqB,EAAE,EAAE,CAChE,IAAA,YAAG,EAAC,OAAO,EAAE,CAAC,EAAE,EAAE,EAAE,CAClB,QAAQ,CAAC,KAAK,CAAC,CACb,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,GAAG,KAAK,GAAG,EAAE,EACrC,GAAG,cAAc,CAClB,CACF,CAAC,CACP,CAAA;QACD,OAAO,WAAW,CAAA;IACpB,CAAC;IAED,YAAY,EAAE,CAAC,CAAS,EAAE,WAAyB,EAAE,EAAE;QACrD,MAAM,gBAAgB,GAAG,iBAAS,CAAC,OAAO,CAAC,WAAW,CAAC,CAAA;QACvD,MAAM,QAAQ,GAAQ,EAAE,CAAA;QACxB,iBAAS,CAAC,OAAO,CACf,CAAC,EAAE,EAAE,GAAG,EAAE,EAAE,CACV,CAAC,QAAQ,CAAC,EAAE,CAAC,GAAG,GAAG,GAAG,gBAAgB,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CACjE,CAAA;QACD,OAAO,QAAQ,CAAA;IACjB,CAAC;CACF,CAAA"}
+13
View File
@@ -0,0 +1,13 @@
export declare class Mean {
private sum;
private _n;
private _min?;
private _max?;
constructor(n?: number, sum?: number);
push(x: number): void;
get n(): number;
get min(): number | undefined;
get max(): number | undefined;
get mean(): number;
clone(): Mean;
}
+34
View File
@@ -0,0 +1,34 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.Mean = void 0;
class Mean {
constructor(n = 0, sum = 0) {
this.sum = sum;
this._min = undefined;
this._max = undefined;
this._n = n;
}
push(x) {
this._n++;
this.sum += x;
this._min = this._min == null || this._min > x ? x : this._min;
this._max = this._max == null || this._max < x ? x : this._max;
}
get n() {
return this._n;
}
get min() {
return this._min;
}
get max() {
return this._max;
}
get mean() {
return this.sum / this.n;
}
clone() {
return new Mean(this.n, this.sum);
}
}
exports.Mean = Mean;
//# sourceMappingURL=Mean.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Mean.js","sourceRoot":"","sources":["../src/Mean.ts"],"names":[],"mappings":";;;AAAA,MAAa,IAAI;IAKf,YACE,CAAC,GAAG,CAAC,EACG,MAAM,CAAC;QAAP,QAAG,GAAH,GAAG,CAAI;QALT,SAAI,GAAY,SAAS,CAAA;QACzB,SAAI,GAAY,SAAS,CAAA;QAM/B,IAAI,CAAC,EAAE,GAAG,CAAC,CAAA;IACb,CAAC;IAED,IAAI,CAAC,CAAS;QACZ,IAAI,CAAC,EAAE,EAAE,CAAA;QACT,IAAI,CAAC,GAAG,IAAI,CAAC,CAAA;QACb,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,IAAI,IAAI,IAAI,IAAI,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAA;QAC9D,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,IAAI,IAAI,IAAI,IAAI,CAAC,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAA;IAChE,CAAC;IAED,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,EAAE,CAAA;IAChB,CAAC;IAED,IAAI,GAAG;QACL,OAAO,IAAI,CAAC,IAAI,CAAA;IAClB,CAAC;IAED,IAAI,GAAG;QACL,OAAO,IAAI,CAAC,IAAI,CAAA;IAClB,CAAC;IAED,IAAI,IAAI;QACN,OAAO,IAAI,CAAC,GAAG,GAAG,IAAI,CAAC,CAAC,CAAA;IAC1B,CAAC;IAED,KAAK;QACH,OAAO,IAAI,IAAI,CAAC,IAAI,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAA;IACnC,CAAC;CACF;AAtCD,oBAsCC"}
+27
View File
@@ -0,0 +1,27 @@
/**
* Aggregate promises efficiently
*/
export declare class Mutex {
private _pushCount;
private readonly _arr;
private get arr();
get pushCount(): number;
push<T>(f: () => Promise<T>): Promise<T>;
/**
* Run f() after all prior-enqueued promises have resolved.
*/
serial<T>(f: () => Promise<T>): Promise<T>;
/**
* Only run f() if all prior have finished, otherwise, no-op and wait until
* all pending have resolved.
*/
runIfIdle<T>(f: () => Promise<T>): undefined | Promise<T>;
get pendingCount(): number;
get pending(): boolean;
get settled(): boolean;
/**
* @return a promise that will be resolved when all previously-pushed Promises
* are resolved. Any promise rejection will throw the whole chain.
*/
awaitAll(): Promise<undefined>;
}
+63
View File
@@ -0,0 +1,63 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.Mutex = void 0;
const Array_1 = require("./Array");
const Deferred_1 = require("./Deferred");
/**
* Aggregate promises efficiently
*/
class Mutex {
constructor() {
this._pushCount = 0;
this._arr = [];
}
get arr() {
(0, Array_1.filterInPlace)(this._arr, (ea) => ea.pending);
return this._arr;
}
get pushCount() {
return this._pushCount;
}
push(f) {
this._pushCount++;
const p = f();
// Don't cause awaitAll to die if a task rejects:
this.arr.push(new Deferred_1.Deferred().observeQuietly(p));
return p;
}
/**
* Run f() after all prior-enqueued promises have resolved.
*/
serial(f) {
return this.push(() => this.awaitAll().then(() => f()));
}
/**
* Only run f() if all prior have finished, otherwise, no-op and wait until
* all pending have resolved.
*/
runIfIdle(f) {
return this.pending ? undefined : this.serial(f);
}
get pendingCount() {
// Don't need vacuuming, so we can use this._arr:
return this._arr.reduce((sum, ea) => sum + (ea.pending ? 1 : 0), 0);
}
get pending() {
return this.pendingCount > 0;
}
get settled() {
// this.arr is a getter that does vacuuming
return this.arr.length === 0;
}
/**
* @return a promise that will be resolved when all previously-pushed Promises
* are resolved. Any promise rejection will throw the whole chain.
*/
awaitAll() {
return this.arr.length === 0
? Promise.resolve(undefined)
: Promise.all(this.arr.map((ea) => ea.promise)).then(() => undefined);
}
}
exports.Mutex = Mutex;
//# sourceMappingURL=Mutex.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Mutex.js","sourceRoot":"","sources":["../src/Mutex.ts"],"names":[],"mappings":";;;AAAA,mCAAuC;AACvC,yCAAqC;AAErC;;GAEG;AACH,MAAa,KAAK;IAAlB;QACU,eAAU,GAAG,CAAC,CAAA;QACL,SAAI,GAAoB,EAAE,CAAA;IAyD7C,CAAC;IAvDC,IAAY,GAAG;QACb,IAAA,qBAAa,EAAC,IAAI,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,OAAO,CAAC,CAAA;QAC5C,OAAO,IAAI,CAAC,IAAI,CAAA;IAClB,CAAC;IAED,IAAI,SAAS;QACX,OAAO,IAAI,CAAC,UAAU,CAAA;IACxB,CAAC;IAED,IAAI,CAAI,CAAmB;QACzB,IAAI,CAAC,UAAU,EAAE,CAAA;QACjB,MAAM,CAAC,GAAG,CAAC,EAAE,CAAA;QACb,iDAAiD;QACjD,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,mBAAQ,EAAE,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,CAAA;QAC/C,OAAO,CAAC,CAAA;IACV,CAAC;IAED;;OAEG;IACH,MAAM,CAAI,CAAmB;QAC3B,OAAO,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,QAAQ,EAAE,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,CAAA;IACzD,CAAC;IAED;;;OAGG;IACH,SAAS,CAAI,CAAmB;QAC9B,OAAO,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,CAAA;IAClD,CAAC;IAED,IAAI,YAAY;QACd,iDAAiD;QACjD,OAAO,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,EAAE,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAA;IACrE,CAAC;IAED,IAAI,OAAO;QACT,OAAO,IAAI,CAAC,YAAY,GAAG,CAAC,CAAA;IAC9B,CAAC;IAED,IAAI,OAAO;QACT,2CAA2C;QAC3C,OAAO,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,CAAC,CAAA;IAC9B,CAAC;IAED;;;OAGG;IACH,QAAQ;QACN,OAAO,IAAI,CAAC,GAAG,CAAC,MAAM,KAAK,CAAC;YAC1B,CAAC,CAAC,OAAO,CAAC,OAAO,CAAC,SAAS,CAAC;YAC5B,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAA;IACzE,CAAC;CACF;AA3DD,sBA2DC"}
+9
View File
@@ -0,0 +1,9 @@
/**
* Only call and return the result of `f` if `obj` is defined (not null nor
* undefined)
*/
export declare function map<T, R>(obj: T | undefined | null, f: (t: T) => R): R | undefined;
export declare function isFunction(obj: any): obj is () => any;
export declare function orElse<T>(obj: T | undefined, defaultValue: T | (() => T)): T;
export declare function fromEntries(arr: [string | undefined, any][]): any;
export declare function omit<T extends Record<string, any>, S extends keyof T>(t: T, ...keysToOmit: S[]): Omit<T, S>;
+42
View File
@@ -0,0 +1,42 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.omit = exports.fromEntries = exports.orElse = exports.isFunction = exports.map = void 0;
/**
* Only call and return the result of `f` if `obj` is defined (not null nor
* undefined)
*/
function map(obj, f) {
return obj != null ? f(obj) : undefined;
}
exports.map = map;
function isFunction(obj) {
return typeof obj === "function";
}
exports.isFunction = isFunction;
function orElse(obj, defaultValue) {
return obj != null
? obj
: isFunction(defaultValue)
? defaultValue()
: defaultValue;
}
exports.orElse = orElse;
function fromEntries(arr) {
const o = {};
for (const [key, value] of arr) {
if (key != null) {
o[key] = value;
}
}
return o;
}
exports.fromEntries = fromEntries;
function omit(t, ...keysToOmit) {
const result = { ...t };
for (const ea of keysToOmit) {
delete result[ea];
}
return result;
}
exports.omit = omit;
//# sourceMappingURL=Object.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Object.js","sourceRoot":"","sources":["../src/Object.ts"],"names":[],"mappings":";;;AAAA;;;GAGG;AACH,SAAgB,GAAG,CACjB,GAAyB,EACzB,CAAc;IAEd,OAAO,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAA;AACzC,CAAC;AALD,kBAKC;AAED,SAAgB,UAAU,CAAC,GAAQ;IACjC,OAAO,OAAO,GAAG,KAAK,UAAU,CAAA;AAClC,CAAC;AAFD,gCAEC;AAED,SAAgB,MAAM,CAAI,GAAkB,EAAE,YAA2B;IACvE,OAAO,GAAG,IAAI,IAAI;QAChB,CAAC,CAAC,GAAG;QACL,CAAC,CAAC,UAAU,CAAC,YAAY,CAAC;YACxB,CAAC,CAAC,YAAY,EAAE;YAChB,CAAC,CAAC,YAAY,CAAA;AACpB,CAAC;AAND,wBAMC;AAED,SAAgB,WAAW,CAAC,GAAgC;IAC1D,MAAM,CAAC,GAAQ,EAAE,CAAA;IACjB,KAAK,MAAM,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,GAAG,EAAE,CAAC;QAC/B,IAAI,GAAG,IAAI,IAAI,EAAE,CAAC;YAChB,CAAC,CAAC,GAAG,CAAC,GAAG,KAAK,CAAA;QAChB,CAAC;IACH,CAAC;IACD,OAAO,CAAC,CAAA;AACV,CAAC;AARD,kCAQC;AAED,SAAgB,IAAI,CAClB,CAAI,EACJ,GAAG,UAAe;IAElB,MAAM,MAAM,GAAG,EAAE,GAAG,CAAC,EAAE,CAAA;IACvB,KAAK,MAAM,EAAE,IAAI,UAAU,EAAE,CAAC;QAC5B,OAAO,MAAM,CAAC,EAAE,CAAC,CAAA;IACnB,CAAC;IACD,OAAO,MAAM,CAAA;AACf,CAAC;AATD,oBASC"}
+24
View File
@@ -0,0 +1,24 @@
/**
* Parser implementations convert stdout and stderr from the underlying child
* process to a more useable format. This can be a no-op passthrough if no
* parsing is necessary.
*/
export interface Parser<T> {
/**
* Invoked once per task.
*
* @param stdout the concatenated stream from `stdin`, stripped of the `PASS`
* or `FAIL` tokens from `BatchProcessOptions`.
*
* @param stderr if defined, includes all text emitted to stderr.
*
* @param passed `true` iff the `PASS` pattern was found in stdout.
*
* @throws an error if the Parser implementation wants to reject the task. It
* is valid to raise Errors if stderr is undefined.
*
* @see BatchProcessOptions
*/
(stdout: string, stderr: string | undefined, passed: boolean): T | Promise<T>;
}
export declare const SimpleParser: Parser<string>;
+13
View File
@@ -0,0 +1,13 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.SimpleParser = void 0;
const String_1 = require("./String");
const SimpleParser = (stdout, stderr, passed) => {
if (!passed)
throw new Error("task failed");
if ((0, String_1.notBlank)(stderr))
throw new Error(stderr);
return stdout;
};
exports.SimpleParser = SimpleParser;
//# sourceMappingURL=Parser.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Parser.js","sourceRoot":"","sources":["../src/Parser.ts"],"names":[],"mappings":";;;AAAA,qCAAmC;AA0B5B,MAAM,YAAY,GAAmB,CAC1C,MAAc,EACd,MAA0B,EAC1B,MAAe,EACf,EAAE;IACF,IAAI,CAAC,MAAM;QAAE,MAAM,IAAI,KAAK,CAAC,aAAa,CAAC,CAAA;IAC3C,IAAI,IAAA,iBAAQ,EAAC,MAAM,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,MAAM,CAAC,CAAA;IAC7C,OAAO,MAAM,CAAA;AACf,CAAC,CAAA;AARY,QAAA,YAAY,gBAQxB"}
+21
View File
@@ -0,0 +1,21 @@
/**
* @param {number} pid process id. Required.
* @returns boolean true if the given process id is in the local process
* table. The PID may be paused or a zombie, though.
*/
export declare function pidExists(pid: number | undefined): boolean;
/**
* Send a signal to the given process id.
*
* @export
* @param pid the process id. Required.
* @param force if true, and the current user has
* permissions to send the signal, the pid will be forced to shut down. Defaults to false.
*/
export declare function kill(pid: number | undefined, force?: boolean): boolean;
/**
* Only used by tests
*
* @returns {Promise<number[]>} all the Process IDs in the process table.
*/
export declare function pids(): Promise<number[]>;
+86
View File
@@ -0,0 +1,86 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.pids = exports.kill = exports.pidExists = void 0;
const node_child_process_1 = __importDefault(require("node:child_process"));
const node_process_1 = __importDefault(require("node:process"));
const Object_1 = require("./Object");
const Platform_1 = require("./Platform");
/**
* @param {number} pid process id. Required.
* @returns boolean true if the given process id is in the local process
* table. The PID may be paused or a zombie, though.
*/
function pidExists(pid) {
if (pid == null || !isFinite(pid) || pid <= 0)
return false;
try {
// signal 0 can be used to test for the existence of a process
// see https://nodejs.org/dist/latest-v18.x/docs/api/process.html#processkillpid-signal
return node_process_1.default.kill(pid, 0);
}
catch (err) {
// We're expecting err.code to be either "EPERM" (if we don't have
// permission to send `pid` and message), or "ESRCH" if that pid doesn't
// exist. EPERM means it _does_ exist!
if ((err === null || err === void 0 ? void 0 : err.code) === "EPERM")
return true;
// failed to get priority--assume the pid is gone.
return false;
}
}
exports.pidExists = pidExists;
/**
* Send a signal to the given process id.
*
* @export
* @param pid the process id. Required.
* @param force if true, and the current user has
* permissions to send the signal, the pid will be forced to shut down. Defaults to false.
*/
function kill(pid, force = false) {
if (pid == null || !isFinite(pid) || pid <= 0)
return false;
try {
return node_process_1.default.kill(pid, force ? "SIGKILL" : undefined);
}
catch (err) {
if (!String(err).includes("ESRCH"))
throw err;
return false;
// failed to get priority--assume the pid is gone.
}
}
exports.kill = kill;
const winRe = /^".+?","(\d+)"/;
const posixRe = /^\s*(\d+)/;
/**
* Only used by tests
*
* @returns {Promise<number[]>} all the Process IDs in the process table.
*/
function pids() {
return new Promise((resolve, reject) => {
node_child_process_1.default.execFile(Platform_1.isWin ? "tasklist" : "ps",
// NoHeader, FOrmat CSV
Platform_1.isWin ? ["/NH", "/FO", "CSV"] : ["-e"], (error, stdout, stderr) => {
if (error != null) {
reject(error);
}
else if (("" + stderr).trim().length > 0) {
reject(new Error(stderr));
}
else
resolve(("" + stdout)
.trim()
.split(/[\n\r]+/)
.map((ea) => ea.match(Platform_1.isWin ? winRe : posixRe))
.map((m) => (0, Object_1.map)(m === null || m === void 0 ? void 0 : m[0], parseInt))
.filter((ea) => ea != null));
});
});
}
exports.pids = pids;
//# sourceMappingURL=Pids.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Pids.js","sourceRoot":"","sources":["../src/Pids.ts"],"names":[],"mappings":";;;;;;AAAA,4EAA8C;AAC9C,gEAAkC;AAClC,qCAA8B;AAC9B,yCAAkC;AAElC;;;;GAIG;AACH,SAAgB,SAAS,CAAC,GAAuB;IAC/C,IAAI,GAAG,IAAI,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,GAAG,IAAI,CAAC;QAAE,OAAO,KAAK,CAAA;IAC3D,IAAI,CAAC;QACH,8DAA8D;QAC9D,uFAAuF;QACvF,OAAO,sBAAO,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC,CAAA;IAC7B,CAAC;IAAC,OAAO,GAAQ,EAAE,CAAC;QAClB,kEAAkE;QAClE,wEAAwE;QACxE,sCAAsC;QACtC,IAAI,CAAA,GAAG,aAAH,GAAG,uBAAH,GAAG,CAAE,IAAI,MAAK,OAAO;YAAE,OAAO,IAAI,CAAA;QAEtC,kDAAkD;QAClD,OAAO,KAAK,CAAA;IACd,CAAC;AACH,CAAC;AAfD,8BAeC;AAED;;;;;;;GAOG;AACH,SAAgB,IAAI,CAAC,GAAuB,EAAE,KAAK,GAAG,KAAK;IACzD,IAAI,GAAG,IAAI,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,GAAG,IAAI,CAAC;QAAE,OAAO,KAAK,CAAA;IAC3D,IAAI,CAAC;QACH,OAAO,sBAAO,CAAC,IAAI,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAA;IACzD,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC;YAAE,MAAM,GAAG,CAAA;QAC7C,OAAO,KAAK,CAAA;QACZ,kDAAkD;IACpD,CAAC;AACH,CAAC;AATD,oBASC;AAED,MAAM,KAAK,GAAG,gBAAgB,CAAA;AAC9B,MAAM,OAAO,GAAG,WAAW,CAAA;AAE3B;;;;GAIG;AACH,SAAgB,IAAI;IAClB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;QACrC,4BAAa,CAAC,QAAQ,CACpB,gBAAK,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI;QACzB,uBAAuB;QACvB,gBAAK,CAAC,CAAC,CAAC,CAAC,KAAK,EAAE,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,EACtC,CAAC,KAAmB,EAAE,MAAc,EAAE,MAAc,EAAE,EAAE;YACtD,IAAI,KAAK,IAAI,IAAI,EAAE,CAAC;gBAClB,MAAM,CAAC,KAAK,CAAC,CAAA;YACf,CAAC;iBAAM,IAAI,CAAC,EAAE,GAAG,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC3C,MAAM,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC,CAAC,CAAA;YAC3B,CAAC;;gBACC,OAAO,CACL,CAAC,EAAE,GAAG,MAAM,CAAC;qBACV,IAAI,EAAE;qBACN,KAAK,CAAC,SAAS,CAAC;qBAChB,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,KAAK,CAAC,gBAAK,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;qBAC9C,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAA,YAAG,EAAC,CAAC,aAAD,CAAC,uBAAD,CAAC,CAAG,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC;qBACjC,MAAM,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,IAAI,IAAI,CAAa,CAC1C,CAAA;QACL,CAAC,CACF,CAAA;IACH,CAAC,CAAC,CAAA;AACJ,CAAC;AAvBD,oBAuBC"}
+3
View File
@@ -0,0 +1,3 @@
export declare const isWin: boolean;
export declare const isMac: boolean;
export declare const isLinux: boolean;
+12
View File
@@ -0,0 +1,12 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.isLinux = exports.isMac = exports.isWin = void 0;
const node_os_1 = __importDefault(require("node:os"));
const _platform = node_os_1.default.platform();
exports.isWin = ["win32", "cygwin"].includes(_platform);
exports.isMac = _platform === "darwin";
exports.isLinux = _platform === "linux";
//# sourceMappingURL=Platform.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Platform.js","sourceRoot":"","sources":["../src/Platform.ts"],"names":[],"mappings":";;;;;;AAAA,sDAAwB;AAExB,MAAM,SAAS,GAAG,iBAAE,CAAC,QAAQ,EAAE,CAAA;AAElB,QAAA,KAAK,GAAG,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAA;AAC/C,QAAA,KAAK,GAAG,SAAS,KAAK,QAAQ,CAAA;AAC9B,QAAA,OAAO,GAAG,SAAS,KAAK,OAAO,CAAA"}
+20
View File
@@ -0,0 +1,20 @@
export declare class Rate {
#private;
readonly periodMs: number;
readonly warmupMs: number;
/**
* @param periodMs the length of time to retain event timestamps for computing
* rate. Events older than this value will be discarded.
* @param warmupMs return `null` from {@link Rate#msPerEvent} if it's been less
* than `warmupMs` since construction or {@link Rate#clear}.
*/
constructor(periodMs?: number, warmupMs?: number);
onEvent(): void;
get eventCount(): number;
get msSinceLastEvent(): number | null;
get msPerEvent(): number | null;
get eventsPerMs(): number;
get eventsPerSecond(): number;
get eventsPerMinute(): number;
clear(): this;
}
+97
View File
@@ -0,0 +1,97 @@
"use strict";
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var _Rate_instances, _Rate_start, _Rate_priorEventTimestamps, _Rate_lastEventTs, _Rate_eventCount, _Rate_vacuum;
Object.defineProperty(exports, "__esModule", { value: true });
exports.Rate = void 0;
const BatchClusterOptions_1 = require("./BatchClusterOptions");
// Implementation notes:
// The prior implementation relied on a weighted average of milliseconds between
// events, which didn't behave well when a series of events happend in the same
// millisecond, and didn't correctly recover if events ceased completely (which
// would be expected if this was an error or timeout rate).
// Keeping each event time in an array makes these calculations more precise,
// but suffers from more memory consumption when measuring high rates and using
// a large periodMs.
class Rate {
/**
* @param periodMs the length of time to retain event timestamps for computing
* rate. Events older than this value will be discarded.
* @param warmupMs return `null` from {@link Rate#msPerEvent} if it's been less
* than `warmupMs` since construction or {@link Rate#clear}.
*/
constructor(periodMs = BatchClusterOptions_1.minuteMs, warmupMs = BatchClusterOptions_1.secondMs) {
_Rate_instances.add(this);
this.periodMs = periodMs;
this.warmupMs = warmupMs;
_Rate_start.set(this, Date.now());
_Rate_priorEventTimestamps.set(this, []);
_Rate_lastEventTs.set(this, null);
_Rate_eventCount.set(this, 0
/**
* @param periodMs the length of time to retain event timestamps for computing
* rate. Events older than this value will be discarded.
* @param warmupMs return `null` from {@link Rate#msPerEvent} if it's been less
* than `warmupMs` since construction or {@link Rate#clear}.
*/
);
}
onEvent() {
var _a;
__classPrivateFieldSet(this, _Rate_eventCount, (_a = __classPrivateFieldGet(this, _Rate_eventCount, "f"), _a++, _a), "f");
const now = Date.now();
__classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").push(now);
__classPrivateFieldSet(this, _Rate_lastEventTs, now, "f");
}
get eventCount() {
return __classPrivateFieldGet(this, _Rate_eventCount, "f");
}
get msSinceLastEvent() {
return __classPrivateFieldGet(this, _Rate_lastEventTs, "f") == null ? null : Date.now() - __classPrivateFieldGet(this, _Rate_lastEventTs, "f");
}
get msPerEvent() {
const msSinceStart = Date.now() - __classPrivateFieldGet(this, _Rate_start, "f");
if (__classPrivateFieldGet(this, _Rate_lastEventTs, "f") == null || msSinceStart < this.warmupMs)
return null;
__classPrivateFieldGet(this, _Rate_instances, "m", _Rate_vacuum).call(this);
const events = __classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").length;
return events === 0 ? null : Math.min(this.periodMs, msSinceStart) / events;
}
get eventsPerMs() {
const mpe = this.msPerEvent;
return mpe == null ? 0 : mpe < 1 ? 1 : 1 / mpe;
}
get eventsPerSecond() {
return this.eventsPerMs * BatchClusterOptions_1.secondMs;
}
get eventsPerMinute() {
return this.eventsPerMs * BatchClusterOptions_1.minuteMs;
}
clear() {
__classPrivateFieldSet(this, _Rate_start, Date.now(), "f");
__classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").length = 0;
__classPrivateFieldSet(this, _Rate_lastEventTs, null, "f");
__classPrivateFieldSet(this, _Rate_eventCount, 0, "f");
return this;
}
}
exports.Rate = Rate;
_Rate_start = new WeakMap(), _Rate_priorEventTimestamps = new WeakMap(), _Rate_lastEventTs = new WeakMap(), _Rate_eventCount = new WeakMap(), _Rate_instances = new WeakSet(), _Rate_vacuum = function _Rate_vacuum() {
const expired = Date.now() - this.periodMs;
const firstValidIndex = __classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").findIndex((ea) => ea > expired);
if (firstValidIndex === -1)
__classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").length = 0;
else if (firstValidIndex > 0) {
__classPrivateFieldGet(this, _Rate_priorEventTimestamps, "f").splice(0, firstValidIndex);
}
};
//# sourceMappingURL=Rate.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Rate.js","sourceRoot":"","sources":["../src/Rate.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;AAAA,+DAA0D;AAE1D,wBAAwB;AAExB,gFAAgF;AAChF,+EAA+E;AAC/E,+EAA+E;AAC/E,2DAA2D;AAE3D,6EAA6E;AAC7E,+EAA+E;AAC/E,oBAAoB;AAEpB,MAAa,IAAI;IAMf;;;;;OAKG;IACH,YACW,WAAW,8BAAQ,EACnB,WAAW,8BAAQ;;QADnB,aAAQ,GAAR,QAAQ,CAAW;QACnB,aAAQ,GAAR,QAAQ,CAAW;QAb9B,sBAAS,IAAI,CAAC,GAAG,EAAE,EAAA;QACV,qCAAkC,EAAE,EAAA;QAC7C,4BAA8B,IAAI,EAAA;QAClC,2BAAc,CAAC;QAEf;;;;;WAKG;UAPY;IAWZ,CAAC;IAEJ,OAAO;;QACL,+CAAA,CAAA,wDAAgB,EAAhB,IAAkB,IAAA,CAAA,MAAA,CAAA;QAClB,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,EAAE,CAAA;QACtB,uBAAA,IAAI,kCAAsB,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;QACpC,uBAAA,IAAI,qBAAgB,GAAG,MAAA,CAAA;IACzB,CAAC;IAaD,IAAI,UAAU;QACZ,OAAO,uBAAA,IAAI,wBAAY,CAAA;IACzB,CAAC;IAED,IAAI,gBAAgB;QAClB,OAAO,uBAAA,IAAI,yBAAa,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,EAAE,GAAG,uBAAA,IAAI,yBAAa,CAAA;IAC1E,CAAC;IAED,IAAI,UAAU;QACZ,MAAM,YAAY,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,uBAAA,IAAI,mBAAO,CAAA;QAC7C,IAAI,uBAAA,IAAI,yBAAa,IAAI,IAAI,IAAI,YAAY,GAAG,IAAI,CAAC,QAAQ;YAAE,OAAO,IAAI,CAAA;QAC1E,uBAAA,IAAI,qCAAQ,MAAZ,IAAI,CAAU,CAAA;QACd,MAAM,MAAM,GAAG,uBAAA,IAAI,kCAAsB,CAAC,MAAM,CAAA;QAChD,OAAO,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,QAAQ,EAAE,YAAY,CAAC,GAAG,MAAM,CAAA;IAC7E,CAAC;IAED,IAAI,WAAW;QACb,MAAM,GAAG,GAAG,IAAI,CAAC,UAAU,CAAA;QAC3B,OAAO,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,GAAG,CAAA;IAChD,CAAC;IAED,IAAI,eAAe;QACjB,OAAO,IAAI,CAAC,WAAW,GAAG,8BAAQ,CAAA;IACpC,CAAC;IAED,IAAI,eAAe;QACjB,OAAO,IAAI,CAAC,WAAW,GAAG,8BAAQ,CAAA;IACpC,CAAC;IAED,KAAK;QACH,uBAAA,IAAI,eAAU,IAAI,CAAC,GAAG,EAAE,MAAA,CAAA;QACxB,uBAAA,IAAI,kCAAsB,CAAC,MAAM,GAAG,CAAC,CAAA;QACrC,uBAAA,IAAI,qBAAgB,IAAI,MAAA,CAAA;QACxB,uBAAA,IAAI,oBAAe,CAAC,MAAA,CAAA;QACpB,OAAO,IAAI,CAAA;IACb,CAAC;CACF;AAvED,oBAuEC;;IA9CG,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,QAAQ,CAAA;IAC1C,MAAM,eAAe,GAAG,uBAAA,IAAI,kCAAsB,CAAC,SAAS,CAC1D,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,GAAG,OAAO,CACrB,CAAA;IACD,IAAI,eAAe,KAAK,CAAC,CAAC;QAAE,uBAAA,IAAI,kCAAsB,CAAC,MAAM,GAAG,CAAC,CAAA;SAC5D,IAAI,eAAe,GAAG,CAAC,EAAE,CAAC;QAC7B,uBAAA,IAAI,kCAAsB,CAAC,MAAM,CAAC,CAAC,EAAE,eAAe,CAAC,CAAA;IACvD,CAAC;AACH,CAAC"}
+3
View File
@@ -0,0 +1,3 @@
/// <reference types="node" />
import { Readable, Writable } from "node:stream";
export declare function destroy(stream: Readable | Writable | null): void;
+18
View File
@@ -0,0 +1,18 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.destroy = void 0;
function destroy(stream) {
var _a;
try {
// .end() may result in an EPIPE when the child process exits. We don't
// care. We just want to make sure the stream is closed.
stream === null || stream === void 0 ? void 0 : stream.removeAllListeners("error");
// It's fine to call .destroy() on a stream that's already destroyed.
(_a = stream === null || stream === void 0 ? void 0 : stream.destroy) === null || _a === void 0 ? void 0 : _a.call(stream);
}
catch {
// don't care
}
}
exports.destroy = destroy;
//# sourceMappingURL=Stream.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Stream.js","sourceRoot":"","sources":["../src/Stream.ts"],"names":[],"mappings":";;;AAEA,SAAgB,OAAO,CAAC,MAAkC;;IACxD,IAAI,CAAC;QACH,uEAAuE;QACvE,wDAAwD;QACxD,MAAM,aAAN,MAAM,uBAAN,MAAM,CAAE,kBAAkB,CAAC,OAAO,CAAC,CAAA;QACnC,qEAAqE;QACrE,MAAA,MAAM,aAAN,MAAM,uBAAN,MAAM,CAAE,OAAO,sDAAI,CAAA;IACrB,CAAC;IAAC,MAAM,CAAC;QACP,aAAa;IACf,CAAC;AACH,CAAC;AAVD,0BAUC"}
+5
View File
@@ -0,0 +1,5 @@
/// <reference types="node" />
export declare function blank(s: string | Buffer | undefined): boolean;
export declare function notBlank(s: string | undefined): s is string;
export declare function ensureSuffix(s: string, suffix: string): string;
export declare function toS(s: any): string;
+21
View File
@@ -0,0 +1,21 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.toS = exports.ensureSuffix = exports.notBlank = exports.blank = void 0;
const Object_1 = require("./Object");
function blank(s) {
return s == null || String(s).trim().length === 0;
}
exports.blank = blank;
function notBlank(s) {
return !blank(s);
}
exports.notBlank = notBlank;
function ensureSuffix(s, suffix) {
return s.endsWith(suffix) ? s : s + suffix;
}
exports.ensureSuffix = ensureSuffix;
function toS(s) {
return s == null ? "" : (0, Object_1.isFunction)(s.toString) ? s.toString() : String(s);
}
exports.toS = toS;
//# sourceMappingURL=String.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"String.js","sourceRoot":"","sources":["../src/String.ts"],"names":[],"mappings":";;;AAAA,qCAAqC;AAErC,SAAgB,KAAK,CAAC,CAA8B;IAClD,OAAO,CAAC,IAAI,IAAI,IAAI,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,CAAA;AACnD,CAAC;AAFD,sBAEC;AAED,SAAgB,QAAQ,CAAC,CAAqB;IAC5C,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;AAClB,CAAC;AAFD,4BAEC;AAED,SAAgB,YAAY,CAAC,CAAS,EAAE,MAAc;IACpD,OAAO,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,MAAM,CAAA;AAC5C,CAAC;AAFD,oCAEC;AAED,SAAgB,GAAG,CAAC,CAAM;IACxB,OAAO,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAA,mBAAU,EAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAA;AAC3E,CAAC;AAFD,kBAEC"}
+38
View File
@@ -0,0 +1,38 @@
/// <reference types="node" />
import { InternalBatchProcessOptions } from "./InternalBatchProcessOptions";
import { Parser } from "./Parser";
type TaskOptions = Pick<InternalBatchProcessOptions, "streamFlushMillis" | "observer" | "passRE" | "failRE" | "logger">;
/**
* Tasks embody individual jobs given to the underlying child processes. Each
* instance has a promise that will be resolved or rejected based on the
* result of the task.
*/
export declare class Task<T = any> {
#private;
readonly command: string;
readonly parser: Parser<T>;
readonly taskId: number;
/**
* @param {string} command is the value written to stdin to perform the given
* task.
* @param {Parser<T>} parser is used to parse resulting data from the
* underlying process to a typed object.
*/
constructor(command: string, parser: Parser<T>);
/**
* @return the resolution or rejection of this task.
*/
get promise(): Promise<T>;
get pending(): boolean;
get state(): string;
onStart(opts: TaskOptions): void;
get runtimeMs(): number | undefined;
toString(): string;
onStdout(buf: string | Buffer): void;
onStderr(buf: string | Buffer): void;
/**
* @return true if the wrapped promise was rejected
*/
reject(error: Error): boolean;
}
export {};
+157
View File
@@ -0,0 +1,157 @@
"use strict";
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var _Task_instances, _Task_opts, _Task_startedAt, _Task_parsing, _Task_settledAt, _Task_d, _Task_stdout, _Task_stderr, _Task_onSettle, _Task_resolve;
Object.defineProperty(exports, "__esModule", { value: true });
exports.Task = void 0;
const Async_1 = require("./Async");
const Deferred_1 = require("./Deferred");
let _taskId = 1;
/**
* Tasks embody individual jobs given to the underlying child processes. Each
* instance has a promise that will be resolved or rejected based on the
* result of the task.
*/
class Task {
/**
* @param {string} command is the value written to stdin to perform the given
* task.
* @param {Parser<T>} parser is used to parse resulting data from the
* underlying process to a typed object.
*/
constructor(command, parser) {
_Task_instances.add(this);
this.command = command;
this.parser = parser;
this.taskId = _taskId++;
_Task_opts.set(this, void 0);
_Task_startedAt.set(this, void 0);
_Task_parsing.set(this, false);
_Task_settledAt.set(this, void 0);
_Task_d.set(this, new Deferred_1.Deferred());
_Task_stdout.set(this, "");
_Task_stderr.set(this, ""
/**
* @param {string} command is the value written to stdin to perform the given
* task.
* @param {Parser<T>} parser is used to parse resulting data from the
* underlying process to a typed object.
*/
);
// We can't use .finally here, because that creates a promise chain that, if
// rejected, results in an uncaught rejection.
__classPrivateFieldGet(this, _Task_d, "f").promise.then(() => __classPrivateFieldGet(this, _Task_instances, "m", _Task_onSettle).call(this), () => __classPrivateFieldGet(this, _Task_instances, "m", _Task_onSettle).call(this));
}
/**
* @return the resolution or rejection of this task.
*/
get promise() {
return __classPrivateFieldGet(this, _Task_d, "f").promise;
}
get pending() {
return __classPrivateFieldGet(this, _Task_d, "f").pending;
}
get state() {
return __classPrivateFieldGet(this, _Task_d, "f").pending
? "pending"
: __classPrivateFieldGet(this, _Task_d, "f").rejected
? "rejected"
: "resolved";
}
onStart(opts) {
__classPrivateFieldSet(this, _Task_opts, opts, "f");
__classPrivateFieldSet(this, _Task_startedAt, Date.now(), "f");
}
get runtimeMs() {
var _a;
return __classPrivateFieldGet(this, _Task_startedAt, "f") == null
? undefined
: ((_a = __classPrivateFieldGet(this, _Task_settledAt, "f")) !== null && _a !== void 0 ? _a : Date.now()) - __classPrivateFieldGet(this, _Task_startedAt, "f");
}
toString() {
return (this.constructor.name +
"(" +
this.command.replace(/\s+/gm, " ").slice(0, 80).trim() +
")#" +
this.taskId);
}
onStdout(buf) {
var _a, _b;
__classPrivateFieldSet(this, _Task_stdout, __classPrivateFieldGet(this, _Task_stdout, "f") + buf.toString(), "f");
const passRE = (_a = __classPrivateFieldGet(this, _Task_opts, "f")) === null || _a === void 0 ? void 0 : _a.passRE;
if (passRE != null && passRE.exec(__classPrivateFieldGet(this, _Task_stdout, "f")) != null) {
// remove the pass token from stdout:
__classPrivateFieldSet(this, _Task_stdout, __classPrivateFieldGet(this, _Task_stdout, "f").replace(passRE, ""), "f");
__classPrivateFieldGet(this, _Task_instances, "m", _Task_resolve).call(this, true);
}
else {
const failRE = (_b = __classPrivateFieldGet(this, _Task_opts, "f")) === null || _b === void 0 ? void 0 : _b.failRE;
if (failRE != null && failRE.exec(__classPrivateFieldGet(this, _Task_stdout, "f")) != null) {
// remove the fail token from stdout:
__classPrivateFieldSet(this, _Task_stdout, __classPrivateFieldGet(this, _Task_stdout, "f").replace(failRE, ""), "f");
__classPrivateFieldGet(this, _Task_instances, "m", _Task_resolve).call(this, false);
}
}
}
onStderr(buf) {
var _a;
__classPrivateFieldSet(this, _Task_stderr, __classPrivateFieldGet(this, _Task_stderr, "f") + buf.toString(), "f");
const failRE = (_a = __classPrivateFieldGet(this, _Task_opts, "f")) === null || _a === void 0 ? void 0 : _a.failRE;
if (failRE != null && failRE.exec(__classPrivateFieldGet(this, _Task_stderr, "f")) != null) {
// remove the fail token from stderr:
__classPrivateFieldSet(this, _Task_stderr, __classPrivateFieldGet(this, _Task_stderr, "f").replace(failRE, ""), "f");
__classPrivateFieldGet(this, _Task_instances, "m", _Task_resolve).call(this, false);
}
}
/**
* @return true if the wrapped promise was rejected
*/
reject(error) {
return __classPrivateFieldGet(this, _Task_d, "f").reject(error);
}
}
exports.Task = Task;
_Task_opts = new WeakMap(), _Task_startedAt = new WeakMap(), _Task_parsing = new WeakMap(), _Task_settledAt = new WeakMap(), _Task_d = new WeakMap(), _Task_stdout = new WeakMap(), _Task_stderr = new WeakMap(), _Task_instances = new WeakSet(), _Task_onSettle = function _Task_onSettle() {
var _a;
__classPrivateFieldSet(this, _Task_settledAt, (_a = __classPrivateFieldGet(this, _Task_settledAt, "f")) !== null && _a !== void 0 ? _a : Date.now(), "f");
}, _Task_resolve = async function _Task_resolve(passed) {
var _a, _b, _c;
// fail always wins.
passed = !__classPrivateFieldGet(this, _Task_d, "f").rejected && passed;
// wait for stderr and stdout to flush:
const flushMs = (_b = (_a = __classPrivateFieldGet(this, _Task_opts, "f")) === null || _a === void 0 ? void 0 : _a.streamFlushMillis) !== null && _b !== void 0 ? _b : 0;
if (flushMs > 0) {
await (0, Async_1.delay)(flushMs);
}
// we're expecting this method may be called concurrently (if there are both
// pass and fail tokens found in stderr and stdout), but we only want to run
// this once, so
if (!this.pending || __classPrivateFieldGet(this, _Task_parsing, "f"))
return;
// this.#opts
// ?.logger()
// .trace("Task.#resolve()", { command: this.command, state: this.state })
// Prevent concurrent parsing:
__classPrivateFieldSet(this, _Task_parsing, true, "f");
try {
const parseResult = await this.parser(__classPrivateFieldGet(this, _Task_stdout, "f"), __classPrivateFieldGet(this, _Task_stderr, "f"), passed);
if (__classPrivateFieldGet(this, _Task_d, "f").resolve(parseResult)) {
}
else {
(_c = __classPrivateFieldGet(this, _Task_opts, "f")) === null || _c === void 0 ? void 0 : _c.observer.emit("internalError", new Error(this.toString() + " ._resolved() more than once"));
}
}
catch (error) {
this.reject(error);
}
};
//# sourceMappingURL=Task.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Task.js","sourceRoot":"","sources":["../src/Task.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;AAAA,mCAA+B;AAC/B,yCAAqC;AASrC,IAAI,OAAO,GAAG,CAAC,CAAA;AAEf;;;;GAIG;AACH,MAAa,IAAI;IAUf;;;;;OAKG;IACH,YACW,OAAe,EACf,MAAiB;;QADjB,YAAO,GAAP,OAAO,CAAQ;QACf,WAAM,GAAN,MAAM,CAAW;QAjBnB,WAAM,GAAG,OAAO,EAAE,CAAA;QAC3B,6BAAmB;QACnB,kCAAmB;QACnB,wBAAW,KAAK,EAAA;QAChB,kCAAmB;QACV,kBAAK,IAAI,mBAAQ,EAAK,EAAA;QAC/B,uBAAU,EAAE,EAAA;QACZ,uBAAU,EAAE;QAEZ;;;;;WAKG;UAPS;QAYV,4EAA4E;QAC5E,8CAA8C;QAC9C,uBAAA,IAAI,eAAG,CAAC,OAAO,CAAC,IAAI,CAClB,GAAG,EAAE,CAAC,uBAAA,IAAI,uCAAU,MAAd,IAAI,CAAY,EACtB,GAAG,EAAE,CAAC,uBAAA,IAAI,uCAAU,MAAd,IAAI,CAAY,CACvB,CAAA;IACH,CAAC;IAED;;OAEG;IACH,IAAI,OAAO;QACT,OAAO,uBAAA,IAAI,eAAG,CAAC,OAAO,CAAA;IACxB,CAAC;IAED,IAAI,OAAO;QACT,OAAO,uBAAA,IAAI,eAAG,CAAC,OAAO,CAAA;IACxB,CAAC;IAED,IAAI,KAAK;QACP,OAAO,uBAAA,IAAI,eAAG,CAAC,OAAO;YACpB,CAAC,CAAC,SAAS;YACX,CAAC,CAAC,uBAAA,IAAI,eAAG,CAAC,QAAQ;gBAChB,CAAC,CAAC,UAAU;gBACZ,CAAC,CAAC,UAAU,CAAA;IAClB,CAAC;IAED,OAAO,CAAC,IAAiB;QACvB,uBAAA,IAAI,cAAS,IAAI,MAAA,CAAA;QACjB,uBAAA,IAAI,mBAAc,IAAI,CAAC,GAAG,EAAE,MAAA,CAAA;IAC9B,CAAC;IAED,IAAI,SAAS;;QACX,OAAO,uBAAA,IAAI,uBAAW,IAAI,IAAI;YAC5B,CAAC,CAAC,SAAS;YACX,CAAC,CAAC,CAAC,MAAA,uBAAA,IAAI,uBAAW,mCAAI,IAAI,CAAC,GAAG,EAAE,CAAC,GAAG,uBAAA,IAAI,uBAAW,CAAA;IACvD,CAAC;IAED,QAAQ;QACN,OAAO,CACL,IAAI,CAAC,WAAW,CAAC,IAAI;YACrB,GAAG;YACH,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE;YACtD,IAAI;YACJ,IAAI,CAAC,MAAM,CACZ,CAAA;IACH,CAAC;IAED,QAAQ,CAAC,GAAoB;;QAC3B,6FAAgB,GAAG,CAAC,QAAQ,EAAE,MAAA,CAAA;QAC9B,MAAM,MAAM,GAAG,MAAA,uBAAA,IAAI,kBAAM,0CAAE,MAAM,CAAA;QACjC,IAAI,MAAM,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,uBAAA,IAAI,oBAAQ,CAAC,IAAI,IAAI,EAAE,CAAC;YACxD,qCAAqC;YACrC,uBAAA,IAAI,gBAAW,uBAAA,IAAI,oBAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,MAAA,CAAA;YAC/C,uBAAA,IAAI,sCAAS,MAAb,IAAI,EAAU,IAAI,CAAC,CAAA;QACrB,CAAC;aAAM,CAAC;YACN,MAAM,MAAM,GAAG,MAAA,uBAAA,IAAI,kBAAM,0CAAE,MAAM,CAAA;YACjC,IAAI,MAAM,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,uBAAA,IAAI,oBAAQ,CAAC,IAAI,IAAI,EAAE,CAAC;gBACxD,qCAAqC;gBACrC,uBAAA,IAAI,gBAAW,uBAAA,IAAI,oBAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,MAAA,CAAA;gBAC/C,uBAAA,IAAI,sCAAS,MAAb,IAAI,EAAU,KAAK,CAAC,CAAA;YACtB,CAAC;QACH,CAAC;IACH,CAAC;IAED,QAAQ,CAAC,GAAoB;;QAC3B,6FAAgB,GAAG,CAAC,QAAQ,EAAE,MAAA,CAAA;QAC9B,MAAM,MAAM,GAAG,MAAA,uBAAA,IAAI,kBAAM,0CAAE,MAAM,CAAA;QACjC,IAAI,MAAM,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,uBAAA,IAAI,oBAAQ,CAAC,IAAI,IAAI,EAAE,CAAC;YACxD,qCAAqC;YACrC,uBAAA,IAAI,gBAAW,uBAAA,IAAI,oBAAQ,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,MAAA,CAAA;YAC/C,uBAAA,IAAI,sCAAS,MAAb,IAAI,EAAU,KAAK,CAAC,CAAA;QACtB,CAAC;IACH,CAAC;IAMD;;OAEG;IACH,MAAM,CAAC,KAAY;QACjB,OAAO,uBAAA,IAAI,eAAG,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;IAC9B,CAAC;CAqCF;AA7ID,oBA6IC;;;IA7CG,yIAAoB,IAAI,CAAC,GAAG,EAAE,MAAA,CAAA;AAChC,CAAC,kBASD,KAAK,wBAAU,MAAe;;IAC5B,oBAAoB;IACpB,MAAM,GAAG,CAAC,uBAAA,IAAI,eAAG,CAAC,QAAQ,IAAI,MAAM,CAAA;IAEpC,uCAAuC;IACvC,MAAM,OAAO,GAAG,MAAA,MAAA,uBAAA,IAAI,kBAAM,0CAAE,iBAAiB,mCAAI,CAAC,CAAA;IAClD,IAAI,OAAO,GAAG,CAAC,EAAE,CAAC;QAChB,MAAM,IAAA,aAAK,EAAC,OAAO,CAAC,CAAA;IACtB,CAAC;IAED,4EAA4E;IAC5E,4EAA4E;IAC5E,gBAAgB;IAChB,IAAI,CAAC,IAAI,CAAC,OAAO,IAAI,uBAAA,IAAI,qBAAS;QAAE,OAAM;IAE1C,aAAa;IACb,aAAa;IACb,0EAA0E;IAE1E,8BAA8B;IAC9B,uBAAA,IAAI,iBAAY,IAAI,MAAA,CAAA;IAEpB,IAAI,CAAC;QACH,MAAM,WAAW,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,uBAAA,IAAI,oBAAQ,EAAE,uBAAA,IAAI,oBAAQ,EAAE,MAAM,CAAC,CAAA;QACzE,IAAI,uBAAA,IAAI,eAAG,CAAC,OAAO,CAAC,WAAW,CAAC,EAAE,CAAC;QACnC,CAAC;aAAM,CAAC;YACN,MAAA,uBAAA,IAAI,kBAAM,0CAAE,QAAQ,CAAC,IAAI,CACvB,eAAe,EACf,IAAI,KAAK,CAAC,IAAI,CAAC,QAAQ,EAAE,GAAG,8BAA8B,CAAC,CAC5D,CAAA;QACH,CAAC;IACH,CAAC;IAAC,OAAO,KAAU,EAAE,CAAC;QACpB,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAA;IACpB,CAAC;AACH,CAAC"}
+2
View File
@@ -0,0 +1,2 @@
export declare const Timeout: unique symbol;
export declare function thenOrTimeout<T>(p: Promise<T>, timeoutMs: number): Promise<T | typeof Timeout>;
+39
View File
@@ -0,0 +1,39 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.thenOrTimeout = exports.Timeout = void 0;
const node_timers_1 = __importDefault(require("node:timers"));
exports.Timeout = Symbol("timeout");
async function thenOrTimeout(p, timeoutMs) {
// TODO: if timeoutMs is [1, 1000], it's probably a mistake. Should we do
// something else in that case?
return timeoutMs <= 1
? p
: new Promise(async (resolve, reject) => {
let pending = true;
try {
const t = node_timers_1.default.setTimeout(() => {
if (pending) {
pending = false;
resolve(exports.Timeout);
}
}, timeoutMs);
const result = await p;
if (pending) {
pending = false;
clearTimeout(t);
resolve(result);
}
}
catch (err) {
if (pending) {
pending = false;
reject(err);
}
}
});
}
exports.thenOrTimeout = thenOrTimeout;
//# sourceMappingURL=Timeout.js.map
+1
View File
@@ -0,0 +1 @@
{"version":3,"file":"Timeout.js","sourceRoot":"","sources":["../src/Timeout.ts"],"names":[],"mappings":";;;;;;AAAA,8DAAgC;AACnB,QAAA,OAAO,GAAG,MAAM,CAAC,SAAS,CAAC,CAAA;AAEjC,KAAK,UAAU,aAAa,CACjC,CAAa,EACb,SAAiB;IAEjB,yEAAyE;IACzE,+BAA+B;IAC/B,OAAO,SAAS,IAAI,CAAC;QACnB,CAAC,CAAC,CAAC;QACH,CAAC,CAAC,IAAI,OAAO,CAAqB,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,EAAE;YACxD,IAAI,OAAO,GAAG,IAAI,CAAA;YAClB,IAAI,CAAC;gBACH,MAAM,CAAC,GAAG,qBAAM,CAAC,UAAU,CAAC,GAAG,EAAE;oBAC/B,IAAI,OAAO,EAAE,CAAC;wBACZ,OAAO,GAAG,KAAK,CAAA;wBACf,OAAO,CAAC,eAAO,CAAC,CAAA;oBAClB,CAAC;gBACH,CAAC,EAAE,SAAS,CAAC,CAAA;gBACb,MAAM,MAAM,GAAG,MAAM,CAAC,CAAA;gBACtB,IAAI,OAAO,EAAE,CAAC;oBACZ,OAAO,GAAG,KAAK,CAAA;oBACf,YAAY,CAAC,CAAC,CAAC,CAAA;oBACf,OAAO,CAAC,MAAM,CAAC,CAAA;gBACjB,CAAC;YACH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,IAAI,OAAO,EAAE,CAAC;oBACZ,OAAO,GAAG,KAAK,CAAA;oBACf,MAAM,CAAC,GAAG,CAAC,CAAA;gBACb,CAAC;YACH,CAAC;QACH,CAAC,CAAC,CAAA;AACR,CAAC;AA9BD,sCA8BC"}
+81
View File
@@ -0,0 +1,81 @@
{
"name": "batch-cluster",
"version": "13.0.0",
"description": "Manage a cluster of child processes",
"main": "dist/BatchCluster.js",
"homepage": "https://photostructure.github.io/batch-cluster.js/",
"bugs": {
"url": "https://github.com/photostructure/batch-cluster.js/issues"
},
"types": "dist/BatchCluster.d.ts",
"repository": {
"type": "git",
"url": "https://github.com/photostructure/batch-cluster.js.git"
},
"engines": {
"node": ">=14"
},
"scripts": {
"ci": "yarn install --frozen-lockfile",
"clean": "rimraf dist",
"prettier": "prettier --write src/*.ts",
"lint": "yarn eslint src --ext .ts",
"compile": "tsc",
"watch": "rimraf dist & tsc --watch",
"pretest": "yarn clean && yarn lint && yarn compile",
"test": "mocha dist/**/*.spec.js",
"docs:1": "typedoc --options .typedoc.js",
"docs:2": "cp .serve.json docs/serve.json",
"docs:3": "touch docs/.nojekyll",
"docs:4": "yarn serve docs",
"docs": "bash -c 'for i in {1..4} ; do yarn docs:$i ; done'"
},
"release-it": {
"src": {
"tagName": "v%s",
"commitArgs": "-S",
"tagArgs": "-S"
},
"hooks": {
"before:init": [
"yarn install",
"yarn lint"
]
},
"github": {
"release": true
}
},
"author": "Matthew McEachen <matthew-batchcluster@photostructure.com>",
"license": "MIT",
"devDependencies": {
"@types/chai": "^4.3.11",
"@types/chai-as-promised": "^7.1.8",
"@types/chai-string": "^1.4.5",
"@types/chai-subset": "^1.3.5",
"@types/mocha": "^10.0.6",
"@types/node": "^20.11.17",
"@typescript-eslint/eslint-plugin": "^6.21.0",
"@typescript-eslint/parser": "^6.21.0",
"chai": "^4.3.10",
"chai-as-promised": "^7.1.1",
"chai-string": "^1.5.0",
"chai-subset": "^1.6.0",
"chai-withintoleranceof": "^1.0.1",
"eslint": "^8.56.0",
"eslint-plugin-import": "^2.29.1",
"mocha": "^10.3.0",
"npm-check-updates": "^16.14.14",
"prettier": "^3.2.5",
"prettier-plugin-organize-imports": "^3.2.4",
"rimraf": "^5.0.5",
"release-it": "^17.0.3",
"seedrandom": "^3.0.5",
"serve": "^14.2.1",
"source-map-support": "^0.5.21",
"split2": "^4.2.0",
"timekeeper": "^2.3.1",
"typedoc": "^0.25.7",
"typescript": "~5.3.3"
}
}
+17
View File
@@ -0,0 +1,17 @@
// Type definitions for chai-withintoleranceof
// Project: https://github.com/RmiTtro/chai-withintoleranceof
// Definitions by: Matthew McEachen <https://github.com/mceachen>
// Definitions: https://github.com/DefinitelyTyped/DefinitelyTyped
/// <reference types="chai" />
interface WithinTolerance {
(expected: number, tol: number | number[], message?: string): Chai.Assertion
}
declare namespace Chai {
interface Assertion extends LanguageChains, NumericComparison, TypeComparison {
withinToleranceOf: WithinTolerance;
withinTolOf: WithinTolerance;
}
}
+5
View File
@@ -0,0 +1,5 @@
{
"name": "@types/chai-withintoleranceof",
"version": "0.0.1",
"types": "./index.d.ts"
}