Remove mocks from AI test files and implement full-stack testing

[devin-ai-integration]

Remove mocks from AI test files and implement full-stack testing

Summary

This PR removes mocks from AI test files (AI.test.tsx, ai.test.ts, and language-models tests) and implements full-stack testing using real AI gateway calls with caching. The changes enable more accurate testing of actual system behavior while maintaining test performance through AI gateway caching.

Key changes:

• Removed vi.mock() calls for ai-functions, ai-providers, and @ai-sdk/openai
• Added setupAITestEnvironment() function for AI gateway configuration
• Updated test expectations to handle real AI responses instead of predictable mock data
• Increased test timeouts to 30 seconds to accommodate real API calls
• Implemented caching mechanism for AI responses during testing
• Fixed ai-providers build issues that were causing module resolution errors

Review & Testing Checklist for Human

⚠️ HIGH RISK - 5 critical items to verify:

• Verify all tests pass in CI with real AI calls - The most critical check since I couldn't run tests locally due to module resolution issues
• Check AI gateway configuration in CI environment - Ensure AI_GATEWAY_URL and AI_GATEWAY_TOKEN are properly set for tests to work
• Test module resolution is fully fixed - Run pnpm test --filter=ai-props to verify ai-providers imports work correctly
• Verify test consistency and reliability - Run tests multiple times to ensure real AI calls don't introduce flakiness
• Check caching mechanism works properly - Verify that repeated test runs use cached responses and don't make redundant API calls

Recommended test plan:

1. Run pnpm test to verify all packages pass
2. Run pnpm test --filter=ai-props multiple times to check for flakiness
3. Check CI logs for any AI gateway configuration issues
4. Monitor test execution times to ensure caching is working

---

Diagram

%%{ init : { "theme" : "default" }}%%
graph TB
    subgraph "AI Props Package"
        AI_test["packages/ai-props/src/<br/>AI.test.tsx"]:::major-edit
        test_utils["packages/ai-props/src/<br/>test-utils.ts"]:::minor-edit
    end
    
    subgraph "AI Database Package"
        ai_db_test["packages/ai-database/src/lib/<br/>__tests__/ai.test.ts"]:::major-edit
    end
    
    subgraph "Language Models Package"
        lang_test["packages/language-models/<br/>tests/index.test.ts"]:::major-edit
    end
    
    subgraph "AI Providers Package"
        provider["packages/ai-providers/src/<br/>provider.ts"]:::minor-edit
        build_js["packages/ai-providers/<br/>build.js"]:::minor-edit
    end
    
    subgraph "AI Functions Package"
        ai_funcs["packages/ai-functions/<br/>test/utils/setupTests.ts"]:::context
    end
    
    AI_test -->|"uses setupAITestEnvironment"| test_utils
    AI_test -->|"imports real functions"| ai_funcs
    ai_db_test -->|"removed @ai-sdk/openai mock"| provider
    lang_test -->|"removed mocks, added setup"| test_utils
    provider -->|"build fixes"| build_js
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit
        L3["Context/No Edit"]:::context
    end
    
    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF

Loading

Notes

• Module resolution challenges: Encountered significant issues with ai-providers package build that caused import failures. Made fixes but this needs verification in CI.
• Performance impact: Tests will be slower with real AI calls, but caching should mitigate repeated API costs.
• Environment requirements: Tests now depend on AI gateway configuration being properly set up.
• Session info: Requested by Nathan (@nathanclevenger) - Link to Devin run

⚠️ Critical: Due to local module resolution issues, I was unable to fully test these changes. The CI environment will be the first real test of whether the mock removal was successful.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring