Postgres on Google Cloud / Lesson 5 of 8

Connecting an application

The proxy was for you. This is for your code — and it is a library, not a process.

≈ 18 minutes · hands on · Python shown in full, others linked

You could ship the proxy as a sidecar and have your application talk to localhost. It works. But there is a better answer for anything you control the source of: a language connector, which does the same job inside your process.

Google describes them as libraries that provide encryption and Identity and Access Management (IAM)-based authorization when connecting to a Cloud SQL instance (Cloud SQL docs). They exist for Java, Python, Go and Node.js, and they need no proxy binary.

Auth Proxy (a process)Connector (a library)
Best for psql, GUI tools, anything whose source you do not control Your own services in a supported language
Deployment A second process to start, supervise and restart A dependency in your manifest
Failure surface Dies independently of your app, and takes connections with it Shares your app's lifecycle
Local plaintext hop Yes, on loopback None — TLS terminates inside your process

The shape of the change

You are deleting configuration, not adding it:

before after ────── ───── DATABASE_HOST INSTANCE_URI = PROJECT_ID:REGION:INSTANCE DATABASE_PORT ────────► DB_NAME = appdb DATABASE_USER DB_USER = app@PROJECT_ID.iam DATABASE_PASSWORD ✗ gone DATABASE_SSLMODE ✗ gone CA certificate ✗ gone

Python, in full

pip install "cloud-sql-python-connector[pg8000]" sqlalchemy
import os
from google.cloud.sql.connector import Connector
import sqlalchemy

connector = Connector()

def getconn():
    return connector.connect(
        os.environ["INSTANCE_URI"],       # PROJECT_ID:REGION:INSTANCE
        "pg8000",
        user=os.environ["DB_USER"],       # app@PROJECT_ID.iam — no suffix
        db=os.environ["DB_NAME"],
        enable_iam_auth=True,             # the flag that removes the password
    )

engine = sqlalchemy.create_engine(
    "postgresql+pg8000://",
    creator=getconn,
    pool_size=5,
    max_overflow=2,
    pool_recycle=1800,                    # see "pooling", below
)

with engine.connect() as conn:
    print(conn.execute(sqlalchemy.text("SELECT current_user")).scalar())

Note the empty URL. SQLAlchemy is told the dialect and handed a creator; the connector supplies the socket. There is no host, no port and no password anywhere in the program.

Where the identity comes from

Connector() with no arguments uses Application Default Credentials — lesson 1. On your laptop that is your gcloud login. On a Google Cloud VM it is the attached service account, with no key involved. Lesson 6 is about the cases where it is neither.

The other three languages

Same three inputs, same flag, different spelling:

LanguageDependencyThe flag
Java cloud-sql-jdbc-socket-factory enableIamAuth=true as a JDBC URL property, alongside cloudSqlInstance and socketFactory
Go cloud-sql-go-connector cloudsqlconn.WithIAMAuthN() as a dialer option
Node.js cloud-sql-nodejs-connector authType: AuthTypes.IAM in the connection options

Not on that list? Use the proxy as a sidecar and connect to 127.0.0.1. It is a completely legitimate answer, and the only one for Ruby, PHP, Rust and .NET.

The three production details

Create the connector once

One Connector per process, at startup, kept for the life of the application. Constructing one per request creates a fresh TLS handshake and token fetch every time, and will fall over under load in a way that looks like the database is slow.

Pooling: the token is not the connection

An open connection is not invalidated when the token that opened it expires — PostgreSQL authenticates at connect time. The connector refreshes tokens so that new connections keep working. What this means in practice:

Egress, again

The connector has the same three destinations as the proxy: sqladmin.googleapis.com:443, oauth2.googleapis.com:443, and the instance on port 3307. A restrictive egress policy that allows 443 and 5432 produces a timeout that looks like a database outage.

Verify as the application, before you switch anything

The check that is worth the ten minutes: connect as the application's own identity, from where the application runs, and see what it can actually do. Not what the grants are believed to say.

SELECT current_user;             -- the service account, not yours
SELECT current_database();       -- the environment you meant
SELECT count(*) FROM some_table; -- reads work
INSERT INTO some_table …;        -- writes work
CREATE TABLE probe (i int);      -- should FAIL, see below

current_database() earns its place. A service reading the wrong environment's data connects successfully, runs successfully, and is wrong. There is no error to catch. Checking which database you are in is the only thing that finds it.

The last line is supposed to fail

A well-designed application role holds no DDL: it can read and write rows and cannot create, alter or drop tables. permission denied for schema public here is the design working. People read it as a broken permission and "fix" it, which is how an application ends up owning its own schema. What to do about migrations instead is lesson 7.

Check yourself

Under load your service throws authentication errors intermittently. Most requests succeed. What is the likely cause?

Intermittent, load-correlated, mostly-fine is the signature of per-request connector construction. A wrong username or blocked port fails every time, not some of the time. Build one connector at startup.

Your service connects and reads happily, but it is reading another environment's data. What catches this?

Nothing errors, so only an explicit assertion finds it. Check current_database() and current_user at startup and refuse to serve if either is wrong. It is three lines and it catches a whole class of configuration mistake.

From memory: what disappears from your configuration when you switch to a connector?

The password, the SSL mode, and any CA certificate. Host and port are replaced by the instance URI. What remains is the instance URI, the database name and the username — none of which are secrets.