Infrastructure & Process Audit

§Executive Summary

0/1Full Running Inventory — What's Actually Live

Top-line counts — 2026-05-24 baseline vs. 2026-08-01 live

Broken (Ready≠True)

Cloud Run Jobs (batch) — all 9

Three jobs (podcast-generator-v1, batch-job-12h, claude-drain-job) have been deployed for 4–7 weeks with zero executions — pure image-storage cost, no compute cost (Cloud Run Jobs only bill when a task actually runs), but worth a deliberate keep-or-delete decision rather than silent accumulation.

Cost posture

The megadoc's cost model (3.6-cost-model.md) is explicit that its figures are estimates pending a real GCP billing export pull — that TODO is still open (tracked as GitHub issue aaas.com#1362, "GCP cost audit — admin-side followup, post-auth"). The one concrete, verified-live cost signal from this audit is the scheduler activation flip above: 42 newly-firing cron jobs is the biggest realistic swing in recurring compute spend since the baseline was written, and it isn't reflected in any cost estimate yet.

2CRO Autoresearch

Can you view its outputs anywhere?

No — and this turns out to be by design, not a bug. The service is stateless: canvas's own proxy (cro_scores_router.py) does a live fetch on every request and returns the JSON synchronously, tagged source: "live". There is no GCS bucket, Firestore collection, or database write path anywhere in the reachable code — nothing persists a run. Close the tab and the result is gone. The only two places it was ever viewable were squr-hub.web.app/cro and aaas-canvas.web.app?module=cro, both live-only dashboards with no history. If you want to actually review past outputs, that requires adding persistence (a Firestore write on each run) — it doesn't exist today.

Making it focus on AaaS domains

Good news: this already works without touching infrastructure. The service takes an optional ?url= query parameter — confirmed both in canvas's proxy code and the frontend widget — and when omitted, falls back to a hardcoded default inside the (SQUR-side, not-in-this-workspace) app code, which resolves live to squr.ai. There is no TARGET_URL/DOMAIN environment variable on the Cloud Run service itself (it carries exactly one env var, GCP_PROJECT=aaas-platform), so infra-level reconfiguration wouldn't do anything anyway.

The real application source (image: europe-west1-docker.pkg.dev/aaas-platform/cro-service/cro-autoresearch) isn't checked out in this sandbox's 13 cloned repos — it's most likely a separate SQUR-side repo. Not needed for the domain-targeting fix above, but would be needed for anything deeper (e.g. changing the hardcoded default itself, or adding persistence).

The 404 discrepancy — now partly explained

Earlier probing of this service (10 different paths, including the two paths visible in same-day access logs) returned 404 on every single one, despite the logs showing live 200s on the exact same revision. Canvas's own ops tool registry has a note that resolves most of this: it lists this service's intended endpoints as /api/evaluate and /api/report, annotated "not yet implemented." None of the paths tested happened to include those two — so the 404s are very likely just hitting routes that were never built, not a broken deployment. Not fully closed — the two paths that did show 200 in the log samples still don't match anything probed, so there may be a third, undocumented working route. Worth a direct request to /api/evaluate and /api/report as the next check.

Cloud Scheduler has no entry for this service anywhere in Terraform or the fleet-cron manifest — it's triggered on-demand only, not on a recurring schedule, which also means it isn't a hidden recurring cost driver.

3 / 5GOOGLE_API_KEY — Secret Manager

Evidence

• The project's own CLAUDE.md documents the AI stack in plain language as "Google AI gemini-2.5-flash direct (free-tier GOOGLE_API_KEY, @google/generative-ai SDK)" — this is the documented, intended architecture, not an accident.
• GitHub issue #1319 ("onboardingSequencePersonalizer missing GOOGLE_API_KEY secret") was deliberately fixed and closed within the last cadence of work — someone recently and intentionally wired this exact secret into a production path that was broken without it.
• Live-referenced across the current Cloud Run/Functions fleet: the secret shows up in run.googleapis.com/secrets annotations on roughly 20 distinct us-central1 revisions, plus key/secret pairs on 13 Cloud Functions v1 entries, plus 2 Cloud Run Jobs (alongside a sibling GEMINI_API_KEY env var).
• Secret Manager's own access audit log shows GOOGLE_API_KEY/versions/latest and /versions/3 being read today (2026-08-01), in the same regular hourly batch as Stripe, GitHub, Resend, and other clearly-live secrets — this is active, current production traffic, not legacy cruft.

What deleting it would actually do

Break AI-dependent functionality across ~20+ live services (enrichment, content generation, onboarding personalization, and more) immediately — Cloud Run resolves secret-backed env vars at container cold-start, so the next cold start after deletion fails outright — while saving $0, because this key is already on the free tier per the documented architecture. That's the opposite of the intended outcome.

What I can't verify from here

I don't have access to the Google AI Studio / Cloud Console billing view for this specific key, so I can't independently confirm its billing tier beyond what the codebase documents and the closed issue implies. If you want a hard guarantee, checking the key's tier directly in Google AI Studio (or the GCP Console's API key details) takes under a minute and would close this out definitively. Separately: note that GOOGLE_API_KEY_SQUR and SQUR_GOOGLE_API_KEY are different secrets entirely and were correctly left untouched — this hold applies only to the plain GOOGLE_API_KEY.

4Billing Reconciliation — Deactivated (corrected mid-audit)

Your instruction ("billing reconciliation can for sure be deactivated, it's not properly set up to start with") applies squarely to this job — an hourly-firing Cloud Function with no working Cloud Run backend behind the name is exactly the kind of not-properly-set-up process you described. Acting on that authorization: the scheduler job firebase-schedule-billingReconciliation-us-central1 was paused (not deleted — fully reversible with one command if you ever want it back: gcloud scheduler jobs resume firebase-schedule-billingReconciliation-us-central1 --location=us-central1 --project=aaas-platform).

Execution-history/error-rate logs for the underlying function couldn't be pulled during this audit — the sandbox's network access to Cloud Logging was timing out repeatedly at the time. The pause itself doesn't depend on that data; it directly stops the hourly trigger regardless of what the function was doing when invoked.

6foundryMegaGrindScheduled — Deep Dive

• Cloud Scheduler's attemptDeadline is 180 seconds. The underlying Cloud Function's actual timeout is 540 seconds.
• Cloud Run keeps CPU allocated for the full lifetime of a request when configured for always-allocated CPU — so when Scheduler gives up and closes the connection at 180s, the function instance is not killed; it keeps executing server-side for up to the remaining ~360 seconds.
• Scheduler logs this as a failed invocation (the caller never got a response), which reads as "broken" from the scheduler's point of view — but the actual grind work behind it can complete successfully regardless.
• IAM confirms the only invoker is the default compute service account (238888064196-compute@developer.gserviceaccount.com via roles/run.invoker) — no external caller, this is purely an internal scheduled job.

7Narrated MP3s — 502 Real Files, but Not Reachable From aaas.diy Itself

The count

A direct listing of gs://aaas-platform-audio/diy/learnings/*.mp3 returns exactly 502 files — not 1,767. This matches apps/diy/public/data/packages.json exactly: 502 of 511 total learning packages carry a populated learnings_audio_url field. A repo-wide search for "1,767" / "1767" found no trace of that number anywhere in code, docs, or generator output — it doesn't match any known package count, MP3 count, or other figure in this project. All 502 are real, substantial audio files (spot-checked 10/10 at 190–370KB each, generated via Google Cloud TTS en-US-Studio-O, 200 audio/mpeg, no 404s or 0-byte placeholders).

8Canvas /api/cro-scores/scores — Root Cause Found

Why it's blank

Canvas loads two kinds of frontend scripts: plain <script src> tags that run synchronously in document order, and type="text/babel" scripts that all batch-execute together at DOMContentLoaded, regardless of where they sit in the HTML. kinds-cro-scores.jsx correctly registers its view into window.VIEWS — but views.jsx, a babel script, does a wholesale window.VIEWS = {...} replace rather than a merge, and because both are babel scripts firing at the same lifecycle event, views.jsx can silently clobber the croScores entry depending on execution order. The tile has no view registered by the time the page tries to render it.

Cost — no action needed

This tile isn't a separately-running service. It's served from a static, vendored JSON file bundled into the existing, shared canvas-api Cloud Run service — the same service that already backs dozens of other canvas views. There is no way to "deactivate" it in isolation, and it carries effectively $0 incremental monthly cost — well under the €1/month threshold in the request. No deactivation needed; only the frontend load-order bug needs fixing.

9Appendix — Full Categorized Service Inventory

Caveat on "orphaned-candidate" flags: this heuristic marks a service as a stale/orphan candidate only from static config (no Cloud Scheduler job targets it, ingress is not restricted, and it is not an event-triggered function). It cannot see actual traffic, webhook callers, or frontend calls, so it systematically over-flags legitimately request-driven services (e.g. core-api-platform, canvas, gtm-sales-intel, and the revenue-generating pics intake service all show up as candidates). Treat the orphaned-candidate list as review targets, not confirmed-dead services.

All 27 categories (click to expand)