Add Functions gRPC core helpers#282
Conversation
Expose low-level protobuf codecs and single work-item execution helpers for Azure Functions Durable JS gRPC consolidation without adding Functions metadata support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expose stable byte-oriented worker processing methods and endpoint/taskHub client options for Azure Functions Durable JS gRPC consolidation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the work-item-executor extraction and base64 protobuf helpers with a minimal, Python-aligned implementation. processOrchestratorRequest and processEntityBatchRequest now reuse the existing worker execution path via an in-process capturing stub (mirroring durabletask-python PR #155's null-stub pattern) instead of refactoring the worker. Removed the base64 helper module, the object-level execute* methods, the V2 EntityRequest host path, and the related exports and tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a minimal “single work-item” execution surface to the Durable Task JS worker so host integrations (e.g., Azure Functions) can execute one orchestrator/entity batch request per invocation without running the long-lived gRPC streaming loop. This aligns with the SDK’s role as a low-level Durable Task Scheduler (sidecar) client/worker implementation.
Changes:
- Added byte-level helper methods on
TaskHubGrpcWorkerto process serializedOrchestratorRequest/EntityBatchRequestpayloads and return serialized responses. - Introduced an in-process
CapturingSidecarStubthat captures completion payloads instead of sending them over gRPC. - Added end-to-end tests and README documentation for the new host-integration surface.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| README.md | Documents the new low-level host integration APIs and clarifies the non–Durable Functions programming model scope. |
| packages/durabletask-js/test/functions-grpc-support.spec.ts | Adds e2e-style tests covering the new byte processors for orchestrations and entities. |
| packages/durabletask-js/src/worker/task-hub-grpc-worker.ts | Implements the new public byte-processing methods and the capturing sidecar stub. |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ator Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t glue Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ease pipeline - Add DurableOrchestrationContext/DurableEntityContext (context.df.* adapters) + wrapOrchestrator/wrapEntity - Add RetryOptions, callHttp (throws), parentInstanceId; align newGuid/callSubOrchestrator with v3 signatures - Add v3 client query-return types (DurableOrchestrationStatus, OrchestrationRuntimeStatus, EntityStateResponse, PurgeHistoryResult) - Add client getStatus/getStatusAll/getStatusBy/readEntityState/purgeInstanceHistory/startNew/waitForCompletionOrCreateCheckStatusResponse - Add deprecated DurableOrchestrationClient alias - Wire host-provided maxGrpcMessageSizeInBytes into gRPC channel options (Python parity) - Remove dead CapturingSidecarStub.abandonRequest field (keep required no-op method) - Add durable-functions package to build/release pipelines
| "dependencies": { | ||
| "@azure/functions": "^4.16.1", | ||
| "@grpc/grpc-js": "^1.14.4", | ||
| "@microsoft/durabletask-js": "0.3.0" |
There was a problem hiding this comment.
TODO: remember to bump this to 0.4.0, and we need to release @microsoft/durabletask-js first
also confirm whether @azure/functions 4.16.1 has the extension changes required,
Drop rootDir, outDir, include, and exclude since they are inherited from ./tsconfig.json. Keep only the baseUrl + paths override that points @microsoft/durabletask-js imports at the built dist output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ment call style and update tests
…to DurableFunctionsClient
…m/microsoft/durabletask-js into pr/282/yunchuwang/functions-grpc-support
… Functions - Introduced `EntityId` class for classic v3 entity identifiers. - Enhanced `DurableEntityContext` to include `isNewlyConstructed` and `entityId` properties. - Implemented `signalEntity` method in `DurableEntityContext` for signaling other entities. - Updated `DurableOrchestrationContext` to track and set custom status. - Added unit tests for new features and behaviors.
…ayload in DurableFunctionsClient
Collapse the 4.0.0-alpha.0 changelog to a single bullet and rewrite the README to focus on what the package supports and why it is needed, dropping the implementation-plan / phase-status / open-questions sections. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…onContext type aliases Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rator Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… context.log to classic orchestration context Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…3-compatible) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…InstanceHistoryBy (v3-compat) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
andystaples
left a comment
There was a problem hiding this comment.
Review summary
Two things ship here: (1) the advertised core gRPC byte-processors (processOrchestratorRequest / processEntityBatchRequest + CapturingSidecarStub) in task-hub-grpc-worker.ts, and (2) an entire new durable-functions@4.0.0-alpha.0 package (packages/azure-functions-durable) — the intended next-major replacement for Azure/azure-functions-durable-js v3.
The core helpers are clean, and reusing the existing execution path with an in-process capturing stub is a nice low-risk approach. The v3 compat adapters (context.df.* for orchestrations/entities and the client method surface) are genuinely broad and well documented.
My main concern is the "drop-in / run unchanged" framing in the README: this is a strong compatibility shim for the common method surface, but several heavily-used v3 patterns and the entire durable-client authoring model are not covered — and there's one silent-failure bug in the orchestrator adapter. Details are inline; grouped here:
Bug (please fix)
wrapOrchestratorpicks classic-vs-native by parameter arity, which silently mis-runs a native single-arg orchestrator (async function*(ctx)) — the body never executes and no error is thrown.
Behavioral breaks vs v3 (worth closing or documenting)
createTimerreturns a non-cancelableTask; theTask.any+timeoutTask.cancel()timeout pattern breaks, and theTaskresult shape changed (isCompleted/isFaulted/result→isComplete/isFailed/getResult()).- Orchestration/entity contexts no longer extend
InvocationContext. - Missing top-level exports:
DummyOrchestrationContext/DummyEntityContext(breaks users' tests), error types (DurableError/AggregatedError/TaskFailedError) forinstanceof,ManagedIdentityTokenSource/TokenSource; theapp.client.*durable-client trigger helpers andsetExceptionPropertiesProviderare also gone. - Client deltas:
getStatusreturns| undefinedand ignoresshowHistory;createCheckStatusResponsenow requires a definedrequest;startNewdrops theversionoption.
Nits
parseJsondoesn't guard""(JSON.parse("")throws).- Core: a version-mismatch abandon makes
processOrchestratorRequestthrow a generic error. - Spurious
deserializeBinaryreorder in the generated proto.d.ts. engines.node >= 22vs@types/node@^18; the Node-22 floor is a notable consumer bump.
Overall a promising foundation — I'd fix the arity bug and soften/annotate the compatibility claims before this reads as a transparent v3 replacement. Leaving this as comments (non-blocking).
| * core-native orchestrator as `(ctx, input)` so it passes through. | ||
| */ | ||
| export function wrapOrchestrator(handler: TOrchestrator | ClassicOrchestrator): TOrchestrator { | ||
| if (typeof handler === "function" && handler.length === 1) { |
There was a problem hiding this comment.
Bug (high): classic-vs-native detection by arity silently mis-runs native single-arg orchestrators.
handler.length === 1 treats any one-parameter function as classic. But a core-native orchestrator written as async function*(ctx) { yield ctx.callActivity(...) } — single param, ignoring input, a very common style — also has arity 1. It gets wrapped and invoked as classic(classicCtx), where classicCtx is { df, log, ... }, so ctx.callActivity is undefined.
Worse: because isGenerator() (line 257) checks for [Symbol.iterator], it returns false for the async generator the handler returns, so the wrapper returns the un-iterated generator and the engine completes the orchestration immediately with garbage output — the body never runs and nothing throws. (Note functions-grpc-support.spec.ts itself writes a native orchestrator as single-arg async (_ctx) => ....)
Suggest detecting by generator kind instead of arity, which matches the engine's actual async-iterator gate: handler.constructor.name === "GeneratorFunction" ⇒ classic (v3 sync generators); AsyncGeneratorFunction / async ⇒ native pass-through. Please also add a round-trip test that drives a native single-arg orchestrator through wrapOrchestrator → the real executor.
| } | ||
|
|
||
| /** Creates a durable timer that fires at the specified time. */ | ||
| createTimer(fireAt: Date | number): Task<unknown> { |
There was a problem hiding this comment.
Break: createTimer returns a non-cancelable core Task, so the canonical durable-timeout pattern no longer works.
v3 createTimer returned a TimerTask with .cancel() / .isCanceled. The ubiquitous pattern
const timeout = context.df.createTimer(deadline);
const work = context.df.callActivity("DoWork");
const winner = yield context.df.Task.any([timeout, work]);
if (!timeout.isCompleted) timeout.cancel();breaks two ways: core Task has no cancel() (→ TypeError, or a leaked timer that keeps the instance alive until it fires), and the result shape changed (isCompleted/isFaulted/result → isComplete/isFailed/getResult()), so timeout.isCompleted is always undefined. This is one of the most common DF patterns — consider surfacing a cancelable timer wrapper, or calling it out prominently in the migration notes.
| } | ||
|
|
||
| /** The object passed to a classic (v3) orchestrator function; its `df` is the durable context. */ | ||
| export interface ClassicOrchestrationContext { |
There was a problem hiding this comment.
Break: the classic context no longer extends InvocationContext.
v3's OrchestrationContext extends InvocationContext, so orchestrators could read context.invocationId, context.functionName, context.extraInputs, etc. This interface exposes only df + log methods, so any orchestrator touching those members won't compile/run. Same applies to ClassicEntityContext in entity-context.ts (just { df }). Worth documenting as a breaking change.
| this.grpcHttpClientTimeout = config.grpcHttpClientTimeout; | ||
| } | ||
|
|
||
| createCheckStatusResponse(request: HttpRequest, instanceId: string): HttpResponse { |
There was a problem hiding this comment.
Subtle break: v3's signature was createCheckStatusResponse(request: HttpRequest | undefined, instanceId) and fell back to the binding's baseUrl when request was undefined. This overload requires a defined request and will throw at request.url (in getInstanceStatusUrl) if called with undefined. Consider accepting HttpRequest | undefined to preserve the v3 contract.
| * @param options - When `showInput` is `false`, input/output payloads are not fetched. | ||
| * @returns The instance status, or `undefined` if the instance does not exist. | ||
| */ | ||
| async getStatus( |
There was a problem hiding this comment.
Subtle break: the return type is DurableOrchestrationStatus | undefined where v3 returned a non-optional DurableOrchestrationStatus, so existing callers doing (await getStatus(id)).runtimeStatus now get a TS error / possible runtime throw. Also GetStatusOptions.showHistory / showHistoryOutput are silently ignored and history is never populated (only showInput is honored). Worth documenting the reduced option set — and note startNew similarly drops v3's version option.
|
|
||
| /** @hidden */ | ||
| function parseJson(value: string | undefined): unknown { | ||
| return value === undefined ? undefined : JSON.parse(value); |
There was a problem hiding this comment.
Nit: parseJson guards undefined but not "". JSON.parse("") throws, so a present-but-empty serialized payload would blow up getStatus / toDurableOrchestrationStatus. Normal flow yields undefined (absent StringValue wrapper) so this is low-risk, but cheap to harden: if (!value) return undefined;.
| export { EntityId } from "./entity-id"; | ||
| export { EntityStateResponse } from "./entity-state-response"; | ||
| export { PurgeHistoryResult } from "./purge-history-result"; | ||
| export { OrchestrationFilter } from "./orchestration-filter"; |
There was a problem hiding this comment.
Break: several v3 top-level exports are missing, so import { ... } from "durable-functions" breaks for:
DummyOrchestrationContext/DummyEntityContext— v3 testing utilities; users' existing unit tests won't compile.TaskFailedError(and v3'sDurableError/AggregatedError) — needed forinstanceofguards on caught orchestration errors; core throws its ownTaskFailedError, which isn't re-exported here.ManagedIdentityTokenSource/TokenSource.
Relatedly, the app surface drops the v3 durable-client trigger helpers (df.app.client.http / timer / storageQueue / ...) and setExceptionPropertiesProvider. The replacement model (generic app.http + df.input.durableClient() + df.getClient(context)) is reasonable, but it changes client-function handler signatures and should be documented as a breaking migration.
| const stub = new CapturingSidecarStub(); | ||
| await this._executeOrchestratorInternal(req, "", stub as unknown as stubs.TaskHubSidecarServiceClient); | ||
|
|
||
| if (!stub.orchestratorResponse) { |
There was a problem hiding this comment.
Minor (core): if versioning is enabled and a version mismatch resolves to abandon, _executeOrchestratorInternal calls the stub's no-op abandonTaskOrchestratorWorkItem and returns without setting a response — so this throws a generic "Orchestrator execution did not produce a response.". Unreachable via the Functions path today (the worker is constructed without versioning), but since this is a public, reusable entry point, consider distinguishing "abandoned" from a genuine failure, or documenting that abandon isn't supported on the single-work-item path.
| static serializeBinaryToWriter(message: OrchestratorRequest, writer: jspb.BinaryWriter): void; | ||
| static deserializeBinary(bytes: Uint8Array): OrchestratorRequest; | ||
| static deserializeBinaryFromReader(message: OrchestratorRequest, reader: jspb.BinaryReader): OrchestratorRequest; | ||
| static deserializeBinary(bytes: Uint8Array): OrchestratorRequest; |
There was a problem hiding this comment.
Nit: this looks like generated-proto churn — deserializeBinary / deserializeBinaryFromReader are merely reordered with no semantic change. Worth dropping from the diff to keep it focused (and confirm the proto codegen version isn't drifting).
| }, | ||
| "homepage": "https://github.com/microsoft/durabletask-js/tree/main/packages/azure-functions-durable#readme", | ||
| "engines": { | ||
| "node": ">=22.0.0" |
There was a problem hiding this comment.
Nit: engines.node >= 22 is a notable consumer-facing floor (drops the Node 18/20 that v3 supported) — worth calling out in the README/changelog. It's also inconsistent with @types/node@^18 in devDependencies.
What
Adds the minimal host-integration surface so Azure Functions can drive a single durable work item per invocation, without running the long-lived gRPC worker loop. This mirrors the Azure Functions consolidation done for Python in durabletask-python PR #155.
API
Two methods on
TaskHubGrpcWorker:processOrchestratorRequest(request: Uint8Array): Promise<Uint8Array>processEntityBatchRequest(request: Uint8Array): Promise<Uint8Array>Each deserializes a TaskHubSidecarService protobuf payload, executes one work item, and returns the serialized response. Host integrations own any transport encoding (for example base64); base64 stays out of the core SDK.
Design (aligned with Python #155)
Rather than refactoring the worker, these methods reuse the worker's existing internal execution path (
_executeOrchestratorInternal/_executeEntityInternal) and pass an in-processCapturingSidecarStubthat records the completion payload instead of sending it over gRPC. This is the same null-stub pattern Python uses (AzureFunctionsNullStub).Deliberately NOT included (kept minimal)
execute*public methods (the byte processors are the single entry point).EntityRequesthost path (not present in Python fix: Reset customStatus on continue-as-new in InMemoryOrchestrationBackend #155; the worker's internal V2 handling for the DTS backend is unchanged).metadataGenerator.Tests
packages/durabletask-js/test/functions-grpc-support.spec.tscovers both byte processors end to end.