Postgres on Google Cloud / Reference

Failure modes

Find the message. Read the layer. Apply the fix.

Start here

Layer 1 fails before the proxy works. Layer 2 fails after the proxy works. Layer 3 fails after you are connected. Several things below are not layers at all — they are network or credential-lifetime problems, and they are the ones people misdiagnose. Full explanation: lesson 4.

The proxy will not start

MessageLayerFix
403 … connectSettings 1 Identity lacks roles/cloudsql.client. Check group membership. If the binding was just created, wait a minute — IAM propagates.
could not find default credentials — Run gcloud auth application-default login. Note the application-default part; plain gcloud auth login does not write this credential.
invalid_rapt / reauth required — A reauthentication policy expired your ADC. It cannot be refreshed non-interactively — a human has to run the login at a terminal.
Cloud SQL Admin API has not been used in project … — Enable it: gcloud services enable sqladmin.googleapis.com.
instance does not exist / malformed name — The URI is PROJECT_ID:REGION:INSTANCE_NAME — three parts, no port, no hostname.
address already in use — Another proxy, or a local PostgreSQL, holds the port. Pick another and re-check which one your client is pointed at.

The proxy runs, but the connection does not

MessageLayerFix
password authentication failed for user "you@example.com" 2 No IAM database user for that identity. Create it with the right --type.
password authentication failed for user "app@….iam.gserviceaccount.com" 2 Drop the .gserviceaccount.com suffix when connecting. Full email when creating the user; truncated when connecting.
Connects, then asks for a password — --auto-iam-authn missing from the proxy command.
pg_hba.conf rejects connection … no encryption
SSL is not enabled on the server		 — Your client has SSL on. The hop to the proxy is plaintext loopback by design. Set SSL mode to disable.
Hangs, then times out — while 443 clearly works — Egress to the instance on port 3307 is blocked. The most commonly missed firewall rule in this whole subject.
server closed the connection unexpectedly — The proxy's credentials expired. It keeps listening, accepts the TCP connection, then drops it. Almost always a proxy left running overnight. Restart it; re-run the ADC login if it complains.
Worked yesterday, fails today, nothing changed — Same as above, nine times in ten. Check this before anything else.
… IAM authentication is not enabled — The instance flag cloudsql.iam_authentication is off. Turning it on restarts the instance — schedule it.

Connected, but you cannot do anything

MessageLayerFix
permission denied for table … 3 Grants missing or stale. Typically a migration or restore created tables the last grant never covered. Re-run the grants script.
permission denied for schema public on CREATE 3 Usually correct behaviour. Application roles hold no DDL. Run migrations as a dedicated DDL role — lesson 7.
must be owner of table … 3 Same thing, for ALTER/DROP. Also correct.
permission denied for sequence … on INSERT 3 Table privileges were granted, sequence USAGE was not. Grant it, and add it to your default privileges.
permission denied to set role 3 Not a member of that role. By design if the role is the object owner.
permission denied for database … on connect 3 No CONNECT privilege on that database. Deliberate if someone revoked it from PUBLIC.

It works, but it is wrong

The dangerous category. Nothing errors.

SymptomCauseFind it with
current_user is not who you expect You are on a different proxy, or a shared password account SELECT current_user, session_user; every session
The data is another environment's Wrong credential for the config — a development key in a staging config, typically SELECT current_database(); asserted at startup
One environment's principal can read another's data Cluster-wide role membership. Roles are shared per instance, privileges are per database The diagonal has_table_privilege query — lesson 7
The application can DROP its own tables It owns them. Ownership grants DDL unconditionally and no revoke changes that Check tableowner in pg_tables — every row should name the owner role, never a login role

Application-only failures

SymptomCause
Intermittent auth errors under load; most requests fine A connector constructed per request. Build one per process at startup.
Auth fails reliably about an hour after deploy A token fetched once by hand instead of letting the connector refresh it.
KeyError / missing credential variable at boot Not running under the secrets injector, or pointed at the wrong config.
Works locally, fails on the server Locally ADC is your login; on the server it is something else. Check which rung of lesson 6 is actually in play.
Worked, then stopped, nothing deployed A key was deleted or the service account disabled. Check the audit log before creating a replacement key.

The routine, when you have no idea