Managed Authentication for AWS RDS and Aurora PostgreSQL

Managed authentication lets the Atatus Infrastructure Agent connect to your RDS or Aurora PostgreSQL database using a short-lived IAM token instead of a password. Nothing long-lived is written to the agent host, and you revoke access by editing an IAM policy rather than by rotating a password that every host holds a copy of. This page covers PostgreSQL on Amazon RDS and Amazon Aurora. If you are using a password today, you can move to managed authentication without changing anything else about your setup.

Before you begin

Check these requirements first, because a mismatch here fails later in a way that is hard to read:

Component	Requirement
Atatus Infra Agent	4.3.0 or higher. Earlier versions ignore the `aws:` block.
Database	Amazon RDS for PostgreSQL, or Amazon Aurora PostgreSQL
Engine	PostgreSQL, and Amazon DocumentDB through a different mechanism. MySQL, SQL Server and Oracle are not supported.
Agent identity	An EC2 instance role, ECS task role, or EKS pod role

Managed authentication requires agent 4.3.0 or higher. An older agent does not report an error when it sees the aws: block. It ignores the block, falls back to password authentication, finds no password, and then fails to authenticate on every attempt. Check your version with atatus-infra-agent version before you start.

How it works

The agent signs a connection token locally using the IAM credentials it already holds, then presents that token to PostgreSQL as the password. The token is valid for 15 minutes and the agent mints a new one for each connection. Two consequences are worth knowing up front:

Token minting is not logged in CloudTrail. The token is signed on the client side, so no AWS API call is made and no event is recorded. You can audit the connection landing in your PostgreSQL logs instead.
The IAM policy controls who, not where. IAM database authentication ignores aws:SourceIp, aws:SourceVpc and aws:SourceVpce in a policy condition. Use security groups to restrict the network path.

Step 1: Turn on IAM database authentication

Your database has to accept IAM tokens before any of the following matters. Without this, the token is signed correctly and then rejected at login. In the AWS console, go to Modify, then Database authentication, and choose Password and IAM database authentication. For a DB instance, you can also use the CLI:

$ aws rds modify-db-instance --db-instance-identifier <DB_IDENTIFIER> \
    --enable-iam-database-authentication --apply-immediately

For an Aurora cluster, the setting lives on the cluster:

$ aws rds modify-db-cluster --db-cluster-identifier <CLUSTER_IDENTIFIER> \
    --enable-iam-database-authentication --apply-immediately

--apply-immediately applies every pending modification on that database, not only this one. Anything queued for the next maintenance window is applied now too, and some of those changes reboot the instance. Check what is pending before you run it:

aws rds describe-db-instances --db-instance-identifier <DB_IDENTIFIER> \
    --query 'DBInstances[0].PendingModifiedValues'

Turning on IAM database authentication is itself a dynamic change and needs no reboot. Leave --apply-immediately off to have it applied in the next maintenance window instead.

Step 2: Create the database role

Connect as your master user and create the atatus role. Run this once per cluster:

CREATE USER atatus WITH LOGIN;
GRANT rds_iam TO atatus;
GRANT pg_monitor TO atatus;
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

GRANT rds_iam is a one-way switch for the role it is applied to. Once a role holds it, that role can never authenticate with a password again, and the only way back is to drop and recreate the role. Never grant it to your master user: every application and operator that connects with the master password is locked out immediately, and you cannot use the master user to undo it. Recovery from that means restoring a snapshot.

pg_monitor is the built-in read-only monitoring role. Without it the agent sees only its own session in pg_stat_activity, and the query text of every other session is redacted to <insufficient privilege>. The agent opens a connection per database it collects from. Roles inherit CONNECT from PUBLIC by default. If you have revoked that, grant it back for each database you want monitored:

GRANT CONNECT ON DATABASE <DB_NAME> TO atatus;

The username in the agent config, the username in the IAM policy, and the PostgreSQL role name must match exactly, including their case. The username is signed into the token, so a case mismatch fails as an authentication error with no hint about the cause.

Step 3: Allow the agent to connect

The identity the agent runs as needs permission to open an IAM-authenticated connection as the atatus role. Attach this policy to the agent’s EC2 instance role, ECS task role, or EKS pod role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AtatusDBMConnect",
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": "arn:aws:rds-db:<REGION>:<ACCOUNT_ID>:dbuser:<RESOURCE_ID>/atatus"
    }
  ]
}

<RESOURCE_ID> is the database resource id, not the database name. Find it under Configuration in the RDS console:

Database type	Resource id format
DB instance	`db-ABCDEFGHIJKLMNOP`
Aurora cluster	`cluster-ABCDEFGHIJKLMNOP`

A restored or cross-region-copied database gets a new resource id. A policy that still names the old id keeps applying without error and silently stops authorizing anything, so the agent starts failing to connect after a restore. Check the resource id whenever you restore from a snapshot.

On ECS Fargate, attach this to the task role, not the execution role. The execution role is used by ECS before your container starts, to pull the image and read secrets. The process inside the container picks up the task role, so that is the identity that signs the token.

Step 4: Configure the agent

Edit /etc/atatus-infra-agent/conf.d/postgresql.d/postgresql.yml and add an aws: block:

metrics:
  - hosts: ["postgres://prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: postgres
    aws:
      region: us-east-2
      instance_endpoint: "prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

These are the fields that matter:

Field	Description
`region`	Region of the database, not of the agent. Required whenever managed authentication is on.
`instance_endpoint`	The endpoint the token is signed for. Defaults to the host in `hosts:`.
`managed_authentication.enabled`	Set to `true` to turn on token authentication.

Remove the password: line when you turn this on. An IAM-authenticated role has no password, and the agent rejects a config carrying both at startup rather than preferring one silently.

Set instance_endpoint explicitly whenever hosts: is not the real RDS endpoint, such as a CNAME, an IP address, or a connection pooler. AWS rejects a token presented to a host it was not signed for. Restart the agent to pick up the change:

$ sudo service atatus-infra-agent restart

This restarts the agent on the host, not your database. It is safe to run at any time: the only effect is a gap in collection of a few seconds, and your databases keep serving throughout. The commands that restart a database are called out separately, and none of them are required by the agent.

TLS

TLS is mandatory under managed authentication and defaults to verify-full. The token is a bearer credential, so a connection that can fall back to plaintext would hand it to anyone on the network path. The agent does not let you turn TLS off here. You do not need to install a certificate. The agent ships the Amazon RDS certificate bundle and trusts it alongside the system store, which matters because the RDS authorities are a private PKI that is in no operating system trust store. verify-ca and require are also accepted. Point ssl_root_cert at a PEM file only if you terminate TLS with your own certificate authority:

    aws:
      region: us-east-2
      managed_authentication:
        enabled: true
      ssl_mode: "verify-full"
      # ssl_root_cert: "/etc/ssl/certs/my-ca.pem"

verify-ca checks the certificate chain but not the server hostname, so it is only as strong as its trust store. Pinned to the Amazon RDS authorities, it still trusts any RDS instance in any AWS account. Prefer verify-full, and use verify-ca only with your own ssl_root_cert.

Monitor several databases with one agent

One agent can collect from any number of PostgreSQL databases. You do not need an agent per database, and the databases do not have to share a region, an account, or a set of credentials. There are three ways to arrange it. They can be mixed in the same setup.

One entry per database

metrics is a list, so add an entry for each database. This is the clearest option and the one to reach for by default, because every setting is scoped to its own database:

metrics:
  - hosts: ["postgres://orders-db.abc123.us-east-2.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: orders
    reported_hostname: orders-production
    labels:
      team: payments
      env: production
    aws:
      region: us-east-2
      instance_endpoint: "orders-db.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

  - hosts: ["postgres://analytics-db.xyz789.eu-west-1.rds.amazonaws.com:5432"]
    username: atatus
    dbm: true
    db_name: analytics
    reported_hostname: analytics-production
    labels:
      team: data
      env: production
    aws:
      region: eu-west-1
      instance_endpoint: "analytics-db.xyz789.eu-west-1.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true

Because each entry carries its own aws block, the databases can sit in different regions, and with role_arn they can sit in different AWS accounts. Each entry can also use a different authentication method, so one database can use a password while another uses IAM tokens.

Set reported_hostname and labels on every entry. Without them, several databases collected by one agent are harder to tell apart in the UI, since they all arrive from the same host.

Several hosts in one entry

When databases share the same credentials and region, list them under a single hosts entry:

metrics:
  - hosts:
      - "postgres://orders-db.abc123.us-east-2.rds.amazonaws.com:5432"
      - "postgres://billing-db.abc123.us-east-2.rds.amazonaws.com:5432"
      - "postgres://reporting-db.abc123.us-east-2.rds.amazonaws.com:5432"
    username: atatus
    dbm: true
    db_name: postgres
    aws:
      region: us-east-2
      managed_authentication:
        enabled: true

Everything outside hosts is shared, so this works only when the username, the region and the authentication method are the same for all of them.

Leave instance_endpoint out when you list several hosts under managed authentication. It is a single value that applies to the whole entry, so setting it makes the agent sign every token for that one endpoint. AWS then rejects the token at every other host, and TLS hostname verification fails as well. With instance_endpoint unset, the agent signs each token for the host it is connecting to, which is what you want here.

Each host still needs its own rds-db:connect permission, because the resource id differs per database. Grant them in one policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AtatusDBMConnect",
      "Effect": "Allow",
      "Action": "rds-db:connect",
      "Resource": [
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:db-ORDERSRESOURCEID/atatus",
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:db-BILLINGRESOURCEID/atatus",
        "arn:aws:rds-db:us-east-2:<ACCOUNT_ID>:dbuser:cluster-REPORTINGRESOURCEID/atatus"
      ]
    }
  ]
}

One file per database

The agent reads every .yml file under conf.d/postgresql.d/, not only postgresql.yml. Give each database its own file when you want them managed separately, for example by different teams or by a configuration management tool:

$ ls /etc/atatus-infra-agent/conf.d/postgresql.d/
orders.yml
analytics.yml
billing.yml
postgresql.yml.template

Each file holds its own metrics list and is loaded independently, so a mistake in one file is easier to find than a mistake in a long combined file.

Only files ending in .yml are read. postgresql.yml.template is ignored because of its suffix, which is why the shipped template does not start collecting on its own. Renaming a file to orders.yml.disabled is a quick way to turn one database off without deleting its config.

Things to watch

Keep these in mind as the number of databases grows:

Every database role has to be set up separately. Steps 2 and 3 above are per database, so create the atatus role and grant rds_iam on each one.
The agent opens a connection per database it collects from. Check max_connections on your smaller instances before you add many databases to one agent.
Security groups are per database. The agent host needs a path to each one on port 5432.
Restart once, after all the edits. The agent reads the whole conf.d tree at startup.

$ sudo service atatus-infra-agent restart

Monitoring a database in another AWS account

RDS has no resource-based policy, so the only workable shape is a role in the database’s account that the agent assumes. The agent’s own credentials are used only to call sts:AssumeRole. The token is then signed with the temporary credentials that role returns. Add role_arn to the managed_authentication block:

    aws:
      region: us-east-2
      instance_endpoint: "prod-pg.abc123.us-east-2.rds.amazonaws.com:5432"
      managed_authentication:
        enabled: true
        role_arn: "arn:aws:iam::<DB_ACCOUNT_ID>:role/AtatusDBMAccess"
        external_id: "<EXTERNAL_ID>"

Set external_id only if the trust policy requires one. In the database owner’s CloudTrail, the agent appears with the role session name atatus-infra-agent. Confirm the assume works from the agent host before you restart the agent:

$ aws sts assume-role \
    --role-arn arn:aws:iam::<DB_ACCOUNT_ID>:role/AtatusDBMAccess \
    --role-session-name atatus-test \
    --external-id <EXTERNAL_ID>

Running the agent outside AWS

With no credential fields set, the agent uses the standard AWS credential chain: environment variables, then the shared config file, then the EC2, ECS or EKS role. That is the best setup inside AWS, because no key is written to disk. On a host outside AWS, supply credentials explicitly. They are used only to sign the token, or to assume the role above:

    aws:
      region: us-east-2
      access_key_id: "<ACCESS_KEY_ID>"
      secret_access_key: "<SECRET_ACCESS_KEY>"
      # credential_profile_name: "atatus"
      # shared_credential_file: "/etc/atatus-infra-agent/aws-credentials"
      managed_authentication:
        enabled: true

Set it up with Terraform

Everything on this page can be provisioned instead of configured by hand: an EC2 host running the agent, the rds-db:connect policy with the resource id resolved at apply time, the security group rule, and optionally the database role itself created at first boot. That has its own page, including the DocumentDB half and how to run both engines from one host:

The working Terraform is at atatus-database-monitoring-sample.

Troubleshooting

Work through these in order. Each step rules out one layer, so you do not have to guess which one is at fault.

Check the agent version

Managed authentication needs 4.3.0 or higher. An older agent ignores the aws: block without complaining:

$ atatus-infra-agent version

Check the agent is running and loaded your config

$ sudo systemctl status atatus-infra-agent --no-pager
$ sudo cat /etc/atatus-infra-agent/conf.d/postgresql.d/postgresql.yml

If more than one agent process is running, the second one exits with a lock error and the first keeps running with whatever config it started with. Stop everything and start once:

$ sudo systemctl stop atatus-infra-agent
$ sudo pkill -f /usr/bin/atatus-infra-agent
$ sudo rm -f /var/lib/atatus-infra-agent/atatus-infra-agent.lock
$ sudo systemctl restart atatus-infra-agent

Turn up the log level

The default log level is warning, which hides the messages that show the integration starting up. Set log_level: info in /etc/atatus-infra-agent/atatus.yml, restart, and read the log:

$ sudo tail -f /var/log/atatus-infra-agent/agent.log

To filter for the PostgreSQL integration:

$ sudo grep -iE 'postgres|rds|token|auth|dbm' /var/log/atatus-infra-agent/agent.log | tail -40

Prove the token works on its own

This is the single most useful check, because it splits the problem cleanly in half. Run it on the agent host, so it uses the same identity the agent uses:

$ curl -sSo /tmp/rds-ca.pem https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
$ TOKEN=$(aws rds generate-db-auth-token --region <REGION> \
    --hostname <ENDPOINT> --port 5432 --username atatus)
$ PGPASSWORD=$TOKEN psql \
    "host=<ENDPOINT> port=5432 dbname=postgres user=atatus sslmode=verify-full sslrootcert=/tmp/rds-ca.pem" \
    -c 'select current_user, pg_is_in_recovery()'

Read the result like this:

What happens	Where the fault is
`generate-db-auth-token` fails	The identity is wrong. On ECS this is almost always the execution role being used where the task role was needed.
The token is created but `psql` hangs	Security group. The token is not involved at all.
`psql` is refused straight away	IAM or the database role. Check the resource id in the policy against the live database, and check the username’s case.
`psql` works but the agent does not	The agent config. Check that `aws.region` is set and that no `password:` key remains.

Query metrics are empty

If the agent connects but you see no queries, pg_stat_statements is most likely not loaded. It is a shared library that has to be loaded when the server starts, so CREATE EXTENSION on its own is not enough:

SHOW shared_preload_libraries;

The output has to list pg_stat_statements. If it does not, set shared_preload_libraries = pg_stat_statements in your parameter group and reboot the instance. This is a static parameter, so applying the parameter group without a reboot does nothing, CREATE EXTENSION still succeeds, and query metrics stay empty with no error.

That reboot restarts your database, not the agent. Every open connection is dropped and the database is unavailable while it comes back, usually for a minute or two. Do it in a maintenance window.On a Multi AZ instance the reboot fails over to the standby unless you pass --no-force-failover, which changes which availability zone your writer is in. On Aurora, reboot the writer instance, and be aware that rebooting the writer triggers a failover.

aws rds reboot-db-instance --db-instance-identifier <WRITER_INSTANCE>

Nothing about the Atatus agent requires this reboot. It is PostgreSQL’s own rule for a static parameter, so if you cannot take the downtime now, everything except query metrics still works.

On Aurora, shared_preload_libraries is set in the DB cluster parameter group, not the DB parameter group. Recent Aurora PostgreSQL versions already include pg_stat_statements in their default cluster parameter group, so check the current value before you create a custom group.

Common error messages

Message	Cause
`PAM authentication failed for user "atatus"`	The database role does not exist, or it does not hold `rds_iam`.
`password authentication failed`	The database does not have IAM database authentication turned on.
`Looks like atatus-infra-agent is already running`	A second agent process. Clear the lock as shown above.
Host metrics arrive but no database data	The agent started before the config file was written. Restart the agent.

​Before you begin

​How it works

​Step 1: Turn on IAM database authentication

​Step 2: Create the database role

​Step 3: Allow the agent to connect

​Step 4: Configure the agent

​TLS

​Monitor several databases with one agent

​One entry per database

​Several hosts in one entry

​One file per database

​Things to watch

​Monitoring a database in another AWS account

​Running the agent outside AWS

​Set it up with Terraform

​Troubleshooting

​Check the agent version

​Check the agent is running and loaded your config

​Turn up the log level

​Prove the token works on its own

​Query metrics are empty

​Common error messages

​Related pages

Before you begin

How it works

Step 1: Turn on IAM database authentication

Step 2: Create the database role

Step 3: Allow the agent to connect

Step 4: Configure the agent

TLS

Monitor several databases with one agent

One entry per database

Several hosts in one entry

One file per database

Things to watch

Monitoring a database in another AWS account

Running the agent outside AWS

Set it up with Terraform

Troubleshooting

Check the agent version

Check the agent is running and loaded your config

Turn up the log level

Prove the token works on its own

Query metrics are empty

Common error messages

Related pages