Skip to content

Offer an optional seven-day cloud trial without a card - #4529

Draft
benjaminshafii wants to merge 12 commits into
devfrom
feat/cloud-week-trial
Draft

Offer an optional seven-day cloud trial without a card#4529
benjaminshafii wants to merge 12 commits into
devfrom
feat/cloud-week-trial

Conversation

@benjaminshafii

@benjaminshafii benjaminshafii commented Sep 5, 2026

Copy link
Copy Markdown
Member

Adds an optional seven-day OpenWork Cloud trial from onboarding and the Web page. Starting requires no card and creates no subscription. Model-provider setup and charges remain separate. Active trials show the exact end date and an optional paid-plan action; expiry retains saved work and blocks new cloud requests without automatic charges.

New trials are off by default behind DEN_OPENWORK_CLOUD_TRIAL_ENABLED. Enable only after releasing compatible desktop clients that accept accessSource: "trial"; disabling the flag later prevents new starts while honoring existing trial dates. The generated SDK now includes the trial routes and access source, fixing the failing generation check.

The trial is durable and one-time per workspace and starting account. Admin-only starts are safe to retry concurrently. The server sends polished reminder and expired-trial emails with the actual end date, retained-work reassurance, and a workspace return action. The returning Web view explains what remains available and makes upgrading optional. Paid or complimentary access suppresses outdated trial notices.

Verification on 09ec4c88084d1d83fa379427ead864f2fac08ccf:

  • Generated SDK CI passed. Local SDK generation check/build, API typecheck, and Den Web typecheck also passed during the final changes.
  • Boundary journey: passed locally: 1/1, zero skips, 69.86s. Proves the disabled rollout creates no trial, admin/member boundaries, concurrent starts, one-time eligibility, real first cloud work, both email phases, expiry against an existing runtime, and paid-access precedence. Reproduce with env -u DATABASE_HOST -u DATABASE_USERNAME -u DATABASE_PASSWORD DB_MODE=mysql pnpm evals:pr specs/remote-session-first-use.test.ts.
  • Browser journey passed in Daytona: 1/1, zero skips, 230.17s. Proves optional skip without writes, no-card start, exact expiry, active/reloaded and expired screens, actual rendered reminder and expired emails, and following the real email link back to Web. Reproduce with env -u DATABASE_HOST -u DATABASE_USERNAME -u DATABASE_PASSWORD OPENWORK_EVAL_REF=feat/cloud-week-trial pnpm evals:e2e cloud-week-trial.
  • Aggregate eval typecheck remains red: a prior clean-base control reproduced 310 diagnostics, with two additional schema-resolution/JSX configuration diagnostics on the trial branch. This check is not claimed as passed.

Current PR verdict: Incomplete only because automated visual judging/publishing requires the expired Infisical login to be refreshed. Both final-head runtime journeys pass; Actual email and returning-Web screenshots are attached for review. The PR stays draft until the five automated visual judgments and their generated evidence are published.

Apply migration 0092 before deployment and follow the rollout order in docs/cloud/free-trial.md. Expiry does not forcibly cancel already-running tasks. Email delivery can repeat after a crash between sending and persisting its marker. The browser fixture has paid billing disabled and makes no live Stripe charge. This PR is independent of the signup redesign; retain the optional CloudTrialCard when combining onboarding changes.

@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated
openwork-app Ignored Ignored Preview Sep 5, 2026 8:17pm UTC
openwork-den Ignored Ignored Preview Sep 5, 2026 8:17pm UTC
openwork-diagnostics Ignored Ignored Preview Sep 5, 2026 8:17pm UTC
openwork-landing Ignored Ignored Preview Sep 5, 2026 8:17pm UTC

@vercel
vercel Bot temporarily deployed to Preview – openwork-landing September 5, 2026 18:20 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-app September 5, 2026 18:20 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-den September 5, 2026 18:20 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics September 5, 2026 18:20 Inactive
Comment on lines +15 to +18

test("an optional seven-day cloud trial explains model access, starts without a card, and ends without a subscription", async ({ world, user, probe, evidence, step }) => {
const trial = async () => {
const result = await probe.api(world.den.admin, "/v1/billing/web-trial");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloud trial email exposes the customer organization name

The new trial email renders the recipient organization's actual name, identifying the customer in outbound content.

Evidence
  • sendDueCloudTrialNotifications() selects the organization name while resolving trial recipients in ee/apps/den-api/src/cloud-trial-notifications.ts:42-48.
  • The notification worker passes that customer-controlled value as organizationName to the cloudTrial email in ee/apps/den-api/src/cloud-trial-notifications.ts:54-60.
  • The new email template renders organizationName in customer-facing content at packages/email/src/templates/cloud-trial.tsx:17 and packages/email/src/templates/cloud-trial.tsx:21.
Also found at 1 additional location
  • packages/email/src/templates/cloud-trial.tsx:16-21

Identified by Warden · confidentiality-review · 5MC-FF8

return {
status: covered ? "ineligible" : trial ? (trial.expires_at.getTime() > Date.now() ? "active" : "expired") : eligible ? "eligible" : "ineligible",
startedAt: trial?.started_at.toISOString() ?? null,
expiresAt: trial?.expires_at.toISOString() ?? null,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trial access can make published desktop clients reject the billing response

When this trial is active, Den reports accessSource: "trial" on the existing web-billing surface, but already-published desktop clients only accept subscription and complimentary, so their access check returns null instead of granting cloud access.

Evidence
  • The new trial record is loaded by loadOpenWorkWebBillingSummary() and passed to resolveOpenWorkWebAccess().
  • Active trials therefore produce accessSource: "trial" on /v1/billing/web.
  • The published desktop parser in apps/app/src/app/lib/den.ts rejects any access source other than subscription, complimentary, or null; the same diff's parser update cannot protect already-published builds.
  • Ship a desktop release that accepts trial before deploying the Den-side trial access behavior; split the Den change until that release is available.
Also found at 4 additional locations
  • ee/apps/den-api/src/openwork-web-access.ts:1-1
  • ee/apps/den-api/src/openwork-web-access.ts:61-61
  • ee/apps/den-api/src/routes/org/billing.ts:165-165
  • ee/apps/den-api/src/stripe-billing.ts:941

Identified by Warden · desktop-den-sync-review · MQK-H9Q

@vercel
vercel Bot temporarily deployed to Preview – openwork-landing September 5, 2026 18:28 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-app September 5, 2026 18:28 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics September 5, 2026 18:28 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-den September 5, 2026 18:28 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics September 5, 2026 18:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-landing September 5, 2026 18:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-app September 5, 2026 18:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-app September 5, 2026 18:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-diagnostics September 5, 2026 18:29 Inactive
@vercel
vercel Bot temporarily deployed to Preview – openwork-landing September 5, 2026 18:29 Inactive
@benjaminshafii

Copy link
Copy Markdown
Member Author

What a person sees when the trial ends

Actual screenshots from the passing browser journey on 09ec4c880, using the real notification worker and email template in an isolated test workspace:

  1. Before expiry: the owner receives the final-day reminder with the actual end date, retained-work reassurance, and no automatic payment.
  2. After expiry: the email says “Your work is still here.” New cloud work is paused; subscribing is an explicit choice.
  3. Returning to Web: the email’s actual link leads to the expired-trial screen, with a clear paid-plan action. The test verifies access is paused, a new free trial is unavailable, and no subscription is created.

The screenshots use an isolated workspace and development email outbox. Billing is deliberately unconfigured in this fixture, so the billing setup notice is not a production checkout claim. No real email or payment was sent.

OPENWORK_EVAL_REF=feat/cloud-week-trial pnpm evals:e2e cloud-week-trial — Daytona, 1 passed, 0 failed, 0 skipped, 230.17s. Earlier attempts exposed two test-harness issues (heading lookup and email links opening a new tab), both corrected. Five automated visual judgments remain pending the expired Infisical login; these captures are review evidence, not a claimed automated visual pass.

Before expiry — actual reminder email

Trial expired — actual email

Returning to OpenWork Web after expiry

@benjaminshafii

benjaminshafii commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Test evidence — a-seven-day-cloud-trial-starts-without-a-card-provisions-the-first-task-once-and — ✅ passed

SHA 09ec4c8 · engine v1

passed · 10 expectations passed · 0 failed · 0 pending

verdict passed · 0 user observations · 0 probes · steps 0/0

ℹ️ ASSERTION — 1. New trials stay unavailable before compatible-client rollout

With Web enabled but the trial rollout disabled, no offer is eligible, a start returns trial_unavailable, and no trial row is created.

  • PASS New trials stay unavailable before compatible-client rollout — With Web enabled but the trial rollout disabled, no offer is eligible, a start returns trial_unavailable, and no trial row is created.

ℹ️ ASSERTION — 2. Paid access is checked before provisioning

A valid write token in an organization without Web access was denied; zero worker rows and zero provider creates.

  • PASS Paid access is checked before provisioning — A valid write token in an organization without Web access was denied; zero worker rows and zero provider creates.

ℹ️ ASSERTION — 3. Only an administrator can start a single seven-day trial without a card or subscription

A member was denied before writes; four concurrent administrator requests shared one persisted seven-day window, created no subscription or runtime, and exposed the same status to members.

  • PASS Only an administrator can start a single seven-day trial without a card or subscription — A member was denied before writes; four concurrent administrator requests shared one persisted seven-day window, created no subscription or runtime, and exposed the same status to members.

ℹ️ ASSERTION — 4. A trial cannot be multiplied across organizations and another team's member cannot start it

The same starter received trial_unavailable in a second organization, a member outside that organization was denied, and its trial table remained empty while the first trial retained its original dates.

  • PASS A trial cannot be multiplied across organizations and another team's member cannot start it — The same starter received trial_unavailable in a second organization, a member outside that organization was denied, and its trial table remained empty while the first trial retained its original dates.

ℹ️ ASSERTION — 5. Only a valid write-scoped create can allocate a workspace

Read-only create, invalid input, read, and send allocated no workers and made no provider creates.

  • PASS Only a valid write-scoped create can allocate a workspace — Read-only create, invalid input, read, and send allocated no workers and made no provider creates.

ℹ️ ASSERTION — 6. Concurrent first requests share one provisioning attempt

Six MCP create calls returned retryable provisioning with no submitted session; one worker row and one Daytona HTTP create were observed, before any browser endpoint was called.

  • PASS Concurrent first requests share one provisioning attempt — Six MCP create calls returned retryable provisioning with no submitted session; one worker row and one Daytona HTTP create were observed, before any browser endpoint was called.

ℹ️ ASSERTION — 7. Retry runs the first task without browser setup, and the browser reuses its workspace

The real provisioner observed healthy HTTP, persisted ready state, and the MCP retry created one native session with the original prompt. A subsequent browser request reused that ready workspace without another sandbox.

  • PASS Retry runs the first task without browser setup, and the browser reuses its workspace — The real provisioner observed healthy HTTP, persisted ready state, and the MCP retry created one native session with the original prompt. A subsequent browser request reused that ready workspace without another sandbox.

ℹ️ ASSERTION — 8. Members get distinct workspaces

A second member's first call created a different worker and sandbox without creating a session on the first member's runtime.

  • PASS Members get distinct workspaces — A second member's first call created a different worker and sandbox without creating a session on the first member's runtime.

ℹ️ ASSERTION — 9. The trial warns its owner, expires once, and blocks new work without renewing or charging

Aged real trial timestamps produced one ending and one expired email, durable sent markers prevented repeats, repeating start retained the expired window, and existing runtime counts stayed unchanged after access denial. No subscription was created.

  • PASS The trial warns its owner, expires once, and blocks new work without renewing or charging — Aged real trial timestamps produced one ending and one expired email, durable sent markers prevented repeats, repeating start retained the expired window, and existing runtime counts stayed unchanged after access denial. No subscription was created.

ℹ️ ASSERTION — 10. An explicit paid upgrade restores access after trial expiry, and canceled paid access still blocks work

The seeded paid entitlement reused the existing runtime for one new session; canceling it denied further work without another sandbox or session.

  • PASS An explicit paid upgrade restores access after trial expiry, and canceled paid access still blocks work — The seeded paid entitlement reused the existing runtime for one new session; canceling it denied further work without another sandbox or session.

Test run created 2026-09-05T20:21:21.638Z · Source: evals/results/test-runs/2026-09-05T20-21-21-638Z-a-seven-day-cloud-trial-starts-without-a-card-provisions-the-first-task-once-and/test-run.json · Repro: pnpm --dir evals artifacts:publish -- --pr 4529 --test-run 2026-09-05T20-21-21-638Z-a-seven-day-cloud-trial-starts-without-a-card-provisions-the-first-task-once-and


Test evidence — an-optional-seven-day-cloud-trial-explains-model-access-starts-without-a-card-an — ✅ passed

SHA 09ec4c8 · engine v1

0/5 screenshots passed · 5 pending · 4 assertions · 4 expectations passed · 0 failed · 5 pending

Dry run: screenshots were not attached.

[world] den(daytona) · [seed] api GET /v1/me/orgs · web(signed in)
[user] see(testId=cloud-trial-card, timeoutMs=90000) · see(text=Try OpenWork Cloud for 7 days) · see(text=/No card required..*You won’t be charged automatically./) · see(text=/[Mm]odel.*separate/)
[probe] api(GET /v1/billing/web-trial) · api(GET /v1/billing/web)
[user] hover(testId=cloud-trial-start)
[vision] looks(1 expectations)
[user] click(testId=cloud-trial-dismiss) · notSee(testId=cloud-trial-card)
[probe] api(GET /v1/billing/web-trial) · api(GET /v1/billing/web)
[user] navigate(/dashboard/web) · see(label=Start 7-day free trial, timeoutMs=90000) · click(label=Start 7-day free trial) · see(text=Your cloud trial is active, timeoutMs=30000)
[probe] api(GET /v1/billing/web-trial)
[user] see(text=/Cloud access ends/)
[probe:raw] [probe:raw] eval()
[user] see(label=Open OpenWork Web)
[probe] api(GET /v1/billing/web)
[user] hover(testId=cloud-trial-card)
[vision] looks(1 expectations)
[user] reload · see(text=Your cloud trial is active, timeoutMs=90000)
[probe] api(GET /v1/billing/web-trial) · api(GET /v1/dev/emails?template=cloudTrial) ×2 · eventually(owner receives the final-day reminder)
[user] navigate(/v1/dev/emails/last) · see(text=Your OpenWork cloud trial ends soon) · see(text=/No payment will be taken/)
[vision] looks(1 expectations)
[user] navigate(/dashboard/web) · see(text=Your cloud trial ends soon)
[probe] api(GET /v1/billing/web)
[user] reload · see(text=Your cloud trial has ended, timeoutMs=90000) · see(label=View paid plan) · notSee(label=Start 7-day free trial)
[probe] api(GET /v1/billing/web-trial) · api(GET /v1/billing/web) · api(GET /v1/dev/emails?template=cloudTrial) · eventually(the expired notification reaches the trial owner)
[user] hover(testId=cloud-trial-card)
[vision] looks(1 expectations)
[user] navigate(/v1/dev/emails/last) · see(text=Your OpenWork cloud trial has ended) · see(text=/No payment will be taken/)
[vision] looks(1 expectations)
[user] see(label=Review cloud access)
[probe:raw] [probe:raw] eval()
[user] navigate(/dashboard/web) · see(text=Your cloud trial has ended, timeoutMs=90000) · notSee(label=Start 7-day free trial) · click(label=View paid plan)
[probe:raw] [probe:raw] eval()
[probe] api(GET /v1/billing/web)
steps 1 ✅ the cloud offer is optional and separates models from cloud access (6.9s) · 2 ✅ starting the trial shows its actual expiry without collecting payment (5.7s) · 3 ✅ the owner receives a clear reminder before cloud access pauses (3.3s) · 4 ✅ expiry explains the next choice and leaves paid access opt-in (13.3s)
verdict passed · 26 user observations (see ×18, looks ×5, notSee ×3) · 19 probes · steps 4/4

⏳ PENDING — 1. The optional cloud trial clearly offers seven days without a card or automatic charges, explains that model access is separate, and provides a way to skip

  • PENDING The optional cloud trial clearly offers seven days without a card or automatic charges, explains that model access is separate, and provides a way to skip — Provider error: Vision validation requires OPENAI_API_KEY or ANTHROPIC_API_KEY, unless ValidateOptions.ask is provided.

ℹ️ ASSERTION — 2. Reviewing and skipping the cloud offer does not start a trial or create a subscription

The visible offer explained the no-card terms and separate model access; after Do this later, the real trial endpoint remained eligible with no dates and billing remained locked with no subscription.

  • PASS Reviewing and skipping the cloud offer does not start a trial or create a subscription — The visible offer explained the no-card terms and separate model access; after Do this later, the real trial endpoint remained eligible with no dates and billing remained locked with no subscription.

⏳ PENDING — 3. The active trial shows when cloud access ends and a clear Open OpenWork Web action, without implying model credits or a paid subscription

  • PENDING The active trial shows when cloud access ends and a clear Open OpenWork Web action, without implying model credits or a paid subscription — Provider error: Vision validation requires OPENAI_API_KEY or ANTHROPIC_API_KEY, unless ValidateOptions.ask is provided.

ℹ️ ASSERTION — 4. The visible start action creates one seven-day trial that survives reload without a paid subscription

{"started":{"status":"active","startedAt":"2026-09-05T20:20:23.394Z","expiresAt":"2026-09-12T20:20:23.394Z"},"hasAccess":true,"subscription":null}

  • PASS The visible start action creates one seven-day trial that survives reload without a paid subscription — {"started":{"status":"active","startedAt":"2026-09-05T20:20:23.394Z","expiresAt":"2026-09-12T20:20:23.394Z"},"hasAccess":true,"subscription":null}

⏳ PENDING — 5. The actual trial reminder email has a readable monochrome OpenWork design, explains when cloud access pauses, reassures that saved work remains, and makes any paid upgrade optional

  • PENDING The actual trial reminder email has a readable monochrome OpenWork design, explains when cloud access pauses, reassures that saved work remains, and makes any paid upgrade optional — Provider error: Vision validation requires OPENAI_API_KEY or ANTHROPIC_API_KEY, unless ValidateOptions.ask is provided.

ℹ️ ASSERTION — 6. The owner is notified before expiry while cloud access remains available

The persisted trial entered its final day; the real notification worker sent the owner an email and Web still reported active access without a subscription.

  • PASS The owner is notified before expiry while cloud access remains available — The persisted trial entered its final day; the real notification worker sent the owner an email and Web still reported active access without a subscription.

⏳ PENDING — 7. The expired trial explains that cloud access has ended and that upgrading is an explicit choice, with no automatic charge and retained work explained

  • PENDING The expired trial explains that cloud access has ended and that upgrading is an explicit choice, with no automatic charge and retained work explained — Provider error: Vision validation requires OPENAI_API_KEY or ANTHROPIC_API_KEY, unless ValidateOptions.ask is provided.

⏳ PENDING — 8. The actual expired-trial email clearly says cloud access has paused, saved work is retained, no payment is taken automatically, and offers a calm way to review cloud access

  • PENDING The actual expired-trial email clearly says cloud access has paused, saved work is retained, no payment is taken automatically, and offers a calm way to review cloud access — Provider error: Vision validation requires OPENAI_API_KEY or ANTHROPIC_API_KEY, unless ValidateOptions.ask is provided.

ℹ️ ASSERTION — 9. Trial expiry appears in the app and sends its owner a notification while access stops and no subscription is created

The real persisted trial expired; the screen offered View paid plan, the development outbox captured its owner notification, and billing stayed locked with subscription null after viewing the plan.

  • PASS Trial expiry appears in the app and sends its owner a notification while access stops and no subscription is created — The real persisted trial expired; the screen offered View paid plan, the development outbox captured its owner notification, and billing stayed locked with subscription null after viewing the plan.

Test run created 2026-09-05T20:16:57.772Z · Source: evals/results/test-runs/2026-09-05T20-16-57-772Z-an-optional-seven-day-cloud-trial-explains-model-access-starts-without-a-card-an/test-run.json · Repro: pnpm --dir evals artifacts:publish -- --pr 4529 --test-run 2026-09-05T20-16-57-772Z-an-optional-seven-day-cloud-trial-explains-model-access-starts-without-a-card-an

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant