CodeCow/stack
1
0
Fork 0
mirror of https://github.com/stack-auth/stack.git synced 2026-06-04 21:04:37 +08:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

[Chore][Fix]: Skip Failing CLI Tests, Rework Failing AI Endpoint Validation test (#1258)

Browse Source 
### Context
In a [previous PR](https://github.com/stack-auth/stack-auth/pull/1258),
we hardcoded the mode to link because local emulator wasn't in a ready
state yet. Soon after, we started encountering three failing tests on
dev

The three cli tests that were failing on dev were failing because they
required the create mode flag to be set. The hardcode to link made the
create paths unreachable. Since we don't have local emulator working,
allowing users to pass in opts.mode would be bad practice- they'd be
triggering local emulator actions without the local emulator being set
up.

Also, there was a failing AI endpoint test. The unified AI endpoint
tests are set up so that if certain env variables are not present,
certain tests aren't run. In practice, if the openrouter key isn't set,
the tests that require actually processing a full AI endpoint result
without forwarding to prod will be skipped. The failing test was meant
to just check schema validation but it performed a full request instead.

### Summary of Changes
We just skip the tests for now. They'll only become relevant when
"create" is a legitimate workflow, which necessitates the function of
local emulator. There is no regression risk because the flow they're
testing isn't active yet, and so the only thing we could possibly test
is that passing the create mode will invoke a certain function which
isn't helpful at this state.

The unified AI endpoint failing test was reworked, another test
accomplishes the same schema validation effect. We don't lose coverage
by axing the failing test because other AI tests already test valid
request bodies (if they weren't valid, they wouldn't get a response).
This commit is contained in:
Aman Ganapathy 2026-03-16 15:35:53 -07:00 committed by GitHub
parent 46cacd4b56
commit db6bd03e06
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 7 additions and 20 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
apps/e2e/tests/backend/endpoints/api/v1/ai-query.test.ts
View File

@ -114,6 +114,8 @@ describe("AI Query Endpoint - Validation", () => {
});
it("rejects invalid tool names", async ({ expect }) => {
// Deterministic non-AI check: this payload is schema-valid, then rejected by
// route-level tool-name validation before any model/provider call.
const response = await niceBackendFetch("/api/v1/ai/query/generate", {
method: "POST",
accessType: "admin",
@ -196,23 +198,6 @@ describe("AI Query Endpoint - Validation", () => {
expect(response.body).toMatchObject({ code: "SCHEMA_ERROR", error: expect.stringContaining("messages") });
});
it("accepts valid request body with all required fields", async ({ expect }) => {
// This will forward to production, so we just verify it doesn't fail validation
const response = await niceBackendFetch("/api/v1/ai/query/generate", {
method: "POST",
accessType: "admin",
body: {
quality: "dumb",
speed: "fast",
tools: [],
systemPrompt: "command-center-ask-ai",
messages: [{ role: "user", content: "test" }],
},
});
expect(response.body).not.toMatchObject({ code: "SCHEMA_ERROR" });
}, 10000); // 60 seconds for AI API call
});
describeWithAi("AI Query Endpoint - Authentication", () => {

8
apps/e2e/tests/general/cli.test.ts
View File

@ -348,7 +348,9 @@ describe("Stack CLI", () => {
// --- init command tests ---
it("init create writes stack.config.ts with selected apps", async ({ expect }) => {
// TODO: Re-enable these create-mode tests once init mode handling is finalized.
// We keep these skipped (instead of todo) so the test logic remains visible and easy to re-enable.
it.skip("init create writes stack.config.ts with selected apps", async ({ expect }) => {
const initDir = path.join(tmpDir, "init-create");
fs.mkdirSync(initDir, { recursive: true });
@ -367,7 +369,7 @@ describe("Stack CLI", () => {
expect(parsed.apps.installed.teams).toEqual({ enabled: true });
});
it("init create with single app", async ({ expect }) => {
it.skip("init create with single app", async ({ expect }) => {
const initDir = path.join(tmpDir, "init-create-single");
fs.mkdirSync(initDir, { recursive: true });
@ -450,7 +452,7 @@ describe("Stack CLI", () => {
expect(stderr).toContain("not found");
});
it("init outputs setup instructions", async ({ expect }) => {
it.skip("init outputs setup instructions", async ({ expect }) => {
const initDir = path.join(tmpDir, "init-instructions");
fs.mkdirSync(initDir, { recursive: true });