Cloud datastore metrics

​

How it works

1. Install ryvn.app/cloud-metrics. The blueprint creates a dedicated collector, a read-only cloud identity, and their config. It runs separately from the platform metrics collector. Install and remove it explicitly; datastore installations do not install it automatically.
2. Declare the resources. Ryvn’s Postgres and Cache modules already emit cloud_metrics and tag resources with the environment ID. Custom Terraform modules must implement the same contract.
3. Choose the metrics. Use the defaults or set the blueprint’s collection input. Ryvn renders an allowlist of the declared resources and selected metrics into the collector config.
4. Check delivery. After the identity and collector deploy, allow a few minutes for samples to arrive. Check the installation’s Metrics tab and the environment’s Grafana; an applied config alone does not prove delivery.

​

Choose what to collect

​

Supported families

​

Default metrics

​

Customize collection

​

Keep the defaults and add a metric

kind: BlueprintInstallation
metadata:
  name: cloud-metrics
spec:
  blueprint: ryvn.app/cloud-metrics
  environment: production
  inputs:
    - name: collection
      value:
        aws:
          rds:
            metrics:
              - name: CPUUtilization
                statistics: [Average]
              - name: FreeStorageSpace
                statistics: [Average]
              - name: ReadIOPS
                statistics: [Average]
              - name: WriteIOPS
                statistics: [Average]
              - name: NetworkReceiveThroughput
                statistics: [Average]
              - name: NetworkTransmitThroughput
                statistics: [Average]
              - name: DatabaseConnections
                statistics: [Maximum]

​

GCP and Azure input values

gcp:
  cloudsql:
    metrics:
      - name: cloudsql.googleapis.com/database/cpu/utilization
      - name: cloudsql.googleapis.com/database/disk/bytes_used
      - name: cloudsql.googleapis.com/database/disk/quota
      - name: cloudsql.googleapis.com/database/network/received_bytes_count
      - name: cloudsql.googleapis.com/database/network/sent_bytes_count
      - name: cloudsql.googleapis.com/database/disk/read_ops_count

azure:
  microsoft.dbforpostgresql/flexibleservers:
    metrics:
      - name: cpu_percent
        aggregations: [Average]
      - name: storage_percent
        aggregations: [Average]

​

Find metrics outside the defaults

• AWS: use exact CloudWatch metric names and statistics: Average, Sum, Minimum, Maximum, or SampleCount.
• GCP: use objects containing only name, set to the full, exact metric type within the family’s domain: cloudsql.googleapis.com/ or redis.googleapis.com/. Scalar strings and additional fields are rejected. Do not use prefixes or wildcards.
• Azure: use exact Azure Monitor metric names and aggregations: Average, Total, Minimum, Maximum, or Count. Custom dimensions selections are rejected because the exporter cannot bound the number of dimension value combinations. Metrics requiring a custom dimension split, such as ShardMemoryUsedBytes, need exporter support before they can be selected. Managed Redis’s built-in InstanceId split is already configured.

​

Replacement, reset, and limits

​

The module contract

variable "ryvn_environment_id" {
  type        = string
  description = "Ryvn environment ID — tags datastore resources for cloud-metrics collection."
}

resource "aws_db_instance" "main" {
  # ... usual configuration ...
  tags = {
    "ryvn.app/cloud-metrics" = var.ryvn_environment_id
  }
}

output "cloud_metrics" {
  value = {
    schema = 1
    targets = {
      this = {
        cloud_resource_id = aws_db_instance.main.arn
      }
    }
  }
}

installations:
  - service: your-datastore-module
    config: |
      ryvn_environment_id: "{{ .ryvn.env.id }}"

• schema is 1 — the contract version.
• targets maps a stable, module-chosen name to each provider object. The name travels onto the collected series as a cloud_target label — keep it short and stable (it must match [a-zA-Z0-9][a-zA-Z0-9_-]{0,62}).
• cloud_resource_id is the provider-native identifier — the full ARN on AWS, the Terraform resource path on GCP (projects/<project>/instances/<name> for Cloud SQL, projects/<project>/locations/<location>/instances/<name> for Memorystore), and the full ARM id on Azure (/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.DBforPostgreSQL/flexibleServers/<name> or Microsoft.Cache/redis). The identifier alone carries the service, resource kind, region, and cloud account; Ryvn resolves the metric family and dimensions from it, so the module never spells a Ryvn-side type or region. For cluster-shaped resources declare every member and the group as separate named targets (for ElastiCache, each cluster ARN plus the replication-group ARN).
• selection is optional provider-specific context for targets whose identifier is not self-describing (e.g. a project-level quota object); datastore identifiers never need it.

​

Permissions: what Ryvn reads in the account

• AWS — an IAM role granting cloudwatch:GetMetricData, cloudwatch:GetMetricStatistics, cloudwatch:ListMetrics, and tag:GetResources, provisioned by the aws-cloud-metrics-identity child. To audit actual reads, enable CloudTrail data-event logging for CloudWatch in the account: GetMetricData and GetMetricStatistics are data events, so they do not appear in a management-events-only trail.
• GCP — the collector’s Kubernetes ServiceAccount is itself the Workload Identity principal; roles/monitoring.viewer is granted on exactly the projects the environment’s targets point into (the union the hub renders as the blueprint’s monitoredIds input).
• Azure — a user-assigned managed identity with Monitoring Reader on the subscriptions the environment’s targets point into (the same monitoredIds seam), bound to the collector’s ServiceAccount by a federated credential.

• The tag (AWS) confines metric-data requests: GetMetricData is issued only for resources the tag admitted. It does not confine enumeration — ListMetrics and tag:GetResources still run across the account in the granted regions, because discovery works by listing candidates and matching the tag. On GCP and Azure the metrics queries themselves enumerate the declared ids (per-project extra_filters; a Resource Graph id in~ filter), so undeclared ids are never requested at all.
• The rendered allowlist and its 50-target cap bound which resources’ telemetry is forwarded to the metrics store — they are scoping rules on emitted series, not on provider API reads.
• Forwarding is default-deny on resource identity: only samples carrying a current admitted resource claim reach the store — exporter self-metrics and anything unrelated never join the cloud-data stream.

​

Per-cloud availability

• GCP — Cloud SQL does publish per-disk read/write ops DELTA metrics (database/disk/read_ops_count / write_ops_count), but they’re left out of the recommended defaults, and the built-in IOPS card’s query only unions the AWS and Azure arms today — selecting the GCP ops metrics makes the stackdriver_ series available to query but the card itself stays empty. Memorystore CPU reports the main thread’s per-window utilization rather than an all-core average (that is the thread Redis bottlenecks on), and Memorystore has no network-throughput metrics, so cache network cards show unavailable.
• Azure — two cache families are supported side by side: legacy Azure Cache for Redis (microsoft.cache/redis) and Azure Managed Redis (microsoft.cache/redisenterprise, keyed by the ARM cluster resource id). Azure Cache doesn’t publish a replication-lag metric — Managed Redis exposes geoReplicationHealthy (0/1 health, not a lag), so the replication-lag card shows unavailable on Azure rather than a misleading lag series. Managed Redis shards report under the per-shard InstanceId dimension, so cache cards sum across shards for a cluster total; cacheRead/cacheWrite remain inbound/outbound respectively (bytes read from / written to the cache). Managed Redis counts (cachehits, cachemisses, evictedkeys, expiredkeys) arrive already per-second — the collector reads them as Average series and the cards take them as-is, unlike legacy Total counts which the cards divide by the scrape window. The InstanceId split is requested with a wildcard (InstanceId eq '*'): the exporter may expand it into more query work than the declared collection estimate — an accepted residual of the stock Azure Monitor exporter, documented rather than capped.
• GCP and Azure — free-space panels need no extra plumbing: Cloud SQL composes its own bytes_used/disk/quota pair and Azure emits storage_free directly.

​

Collection status

• Pending apply — when the rendered collector config differs from what the blueprint install currently carries, the status shows pending apply (CLI) and the Metrics tab notes it. A collection change, a new declaration, or a resource removal all render a new config; the flag clears once the collector’s reconcile applies it. It is informational, not an error — the collector keeps serving the previously applied selection in the meantime.
• Family selection — each resolved family reports enabled or disabled. Disabled means the family is present in declarations but the collection selection turned it off (metrics: []); the CLI shows the disabled family names next to the status.

​

Where the metrics live

• You query them through that environment’s Grafana (or any client pointed at its store); there is no cross-environment rollup of cloud metrics.
• An environment without the observability stack produces no queryable series — the collector has nowhere to write. Because the hub never queries the destination, installation status stays unverified whether or not series actually land; check the environment’s Grafana to confirm delivery.

​

Dashboards and queries

• A card whose prerequisite metric isn’t in your selection shows empty/unavailable rather than silently widening the collector or fabricating zeroes.
• Custom metrics you add don’t automatically acquire cards — Grafana and direct queries are the advanced path, and they work for any native name the collector forwarded.

min by (cloud_resource_id) (aws_rds_free_storage_space_average{environment="production", installation="orders-db"})