Postgres on Google Cloud / Lesson 5 of 8
The proxy was for you. This is for your code — and it is a library, not a process.
You could ship the proxy as a sidecar and have your application talk to
localhost. It works. But there is a better answer for anything you
control the source of: a language connector, which does the same
job inside your process.
Google describes them as libraries that provide encryption and Identity and
Access Management (IAM)-based authorization when connecting to a Cloud SQL
instance
(Cloud
SQL docs). They exist for Java, Python, Go and Node.js, and
they need no proxy binary.
| Auth Proxy (a process) | Connector (a library) | |
|---|---|---|
| Best for | psql, GUI tools, anything whose source you do not control |
Your own services in a supported language |
| Deployment | A second process to start, supervise and restart | A dependency in your manifest |
| Failure surface | Dies independently of your app, and takes connections with it | Shares your app's lifecycle |
| Local plaintext hop | Yes, on loopback | None — TLS terminates inside your process |
You are deleting configuration, not adding it:
pip install "cloud-sql-python-connector[pg8000]" sqlalchemy
import os
from google.cloud.sql.connector import Connector
import sqlalchemy
connector = Connector()
def getconn():
return connector.connect(
os.environ["INSTANCE_URI"], # PROJECT_ID:REGION:INSTANCE
"pg8000",
user=os.environ["DB_USER"], # app@PROJECT_ID.iam — no suffix
db=os.environ["DB_NAME"],
enable_iam_auth=True, # the flag that removes the password
)
engine = sqlalchemy.create_engine(
"postgresql+pg8000://",
creator=getconn,
pool_size=5,
max_overflow=2,
pool_recycle=1800, # see "pooling", below
)
with engine.connect() as conn:
print(conn.execute(sqlalchemy.text("SELECT current_user")).scalar())
Note the empty URL. SQLAlchemy is told the dialect and handed a
creator; the connector supplies the socket. There is no host, no
port and no password anywhere in the program.
Connector() with no arguments uses Application Default
Credentials — lesson 1. On your laptop
that is your gcloud login. On a Google Cloud VM it is the attached
service account, with no key involved.
Lesson 6 is about the cases where it is
neither.
Same three inputs, same flag, different spelling:
| Language | Dependency | The flag |
|---|---|---|
| Java | cloud-sql-jdbc-socket-factory | enableIamAuth=true as a JDBC URL property, alongside
cloudSqlInstance and socketFactory |
| Go | cloud-sql-go-connector | cloudsqlconn.WithIAMAuthN() as a dialer option |
| Node.js | cloud-sql-nodejs-connector | authType: AuthTypes.IAM in the connection options |
Not on that list? Use the proxy as a sidecar and connect to
127.0.0.1. It is a completely legitimate answer, and the only one
for Ruby, PHP, Rust and .NET.
One Connector per process, at startup, kept for the life of the
application. Constructing one per request creates a fresh TLS handshake and token
fetch every time, and will fall over under load in a way that looks like the
database is slow.
An open connection is not invalidated when the token that opened it expires — PostgreSQL authenticates at connect time. The connector refreshes tokens so that new connections keep working. What this means in practice:
pool_recycle (or your pool's equivalent) to something
under an hour anyway. It costs nothing and it means a connection that has gone
quietly bad gets replaced.The connector has the same three destinations as the proxy:
sqladmin.googleapis.com:443,
oauth2.googleapis.com:443, and the instance on port
3307. A restrictive egress policy that allows 443 and 5432 produces a
timeout that looks like a database outage.
The check that is worth the ten minutes: connect as the application's own identity, from where the application runs, and see what it can actually do. Not what the grants are believed to say.
SELECT current_user; -- the service account, not yours
SELECT current_database(); -- the environment you meant
SELECT count(*) FROM some_table; -- reads work
INSERT INTO some_table …; -- writes work
CREATE TABLE probe (i int); -- should FAIL, see below
current_database() earns its place. A service reading the wrong
environment's data connects successfully, runs successfully, and is wrong. There
is no error to catch. Checking which database you are in is the only thing that
finds it.
A well-designed application role holds no DDL: it can read and write rows
and cannot create, alter or drop tables. permission denied for schema
public here is the design working. People read it as a broken permission
and "fix" it, which is how an application ends up owning its own schema. What
to do about migrations instead is lesson
7.
Under load your service throws authentication errors intermittently. Most requests succeed. What is the likely cause?
Intermittent, load-correlated, mostly-fine is the signature of per-request connector construction. A wrong username or blocked port fails every time, not some of the time. Build one connector at startup.
Your service connects and reads happily, but it is reading another environment's data. What catches this?
Nothing errors, so only an explicit assertion finds it. Check
current_database() and current_user at startup and
refuse to serve if either is wrong. It is three lines and it catches a whole
class of configuration mistake.
From memory: what disappears from your configuration when you switch to a connector?
The password, the SSL mode, and any CA certificate. Host and port are replaced by the instance URI. What remains is the instance URI, the database name and the username — none of which are secrets.