Skip to main content
A dedicated LGTM installation can expose Grafana, Loki, and Mimir to people on your tailnet. Ryvn creates three HTTPS Ingresses and configures Grafana’s login URL. Your tailnet administrator controls who can reach them. Collectors continue to send logs and metrics through the environment’s internal gateway. Enabling Tailscale does not change their connection URLs.

Before you enable access

You need:
  • A dedicated observability-infra blueprint installation with Stack Exposure set to Private. Use a blueprint and tailscale-resources chart release that supports the inputs below. If the inputs are missing, contact your Ryvn administrator to update the releases.
  • A running Tailscale Kubernetes Operator in the same environment, with the ProxyGroup and ProxyClass CRDs and the tailscale IngressClass. HA ingress requires Operator 1.84.0 or later; use a current stable operator with matching proxy images, CRDs, and Kubernetes permissions.
  • Access to the tailnet’s DNS settings, trust credentials, and access policy, or an administrator who can update them.
  • A Tailscale client connected to that tailnet on each user’s device.
Switching Stack Exposure from Public to Private disables public endpoints and changes collector URLs to the environment’s internal domain. First verify private connectivity, DNS, and a trusted HTTPS certificate on the internal gateway. When using private links, configure Private Service Publisher in the LGTM environment and Private Service Consumer in each collector environment before switching exposure.
If the operator is not installed, follow the operator setup instructions. LGTM uses the existing operator; it does not install another one.

1. Configure the tailnet

DNS and operator credentials

Enable MagicDNS and HTTPS certificates. Copy the tailnet’s DNS suffix, such as example.ts.net. This becomes tailnetDomain; it is specific to your tailnet. The operator’s OAuth or workload identity credential needs write access to General/Services, Devices/Core, and Keys/Auth Keys, with the appropriate operator tag. Ensure the operator owns the tags used for proxy devices. An operator that already exposes a database or the Kubernetes API may still lack Services permission. See Tailscale’s credential setup.

Service advertisement and user access

Allow the proxy devices to advertise the new Tailscale Services, then grant your users access to those Services on TCP 443. These are separate permissions. The following fragment assumes the default tag:k8s tag and an existing group:observability. It grants that group access to the three named Services. Merge it into your existing policy and replace the group, tags, and prod-lgtm prefix with your own values.
In autoApprovers.services, the key is a Service tag and the values identify the proxy devices allowed to advertise it. The example approves advertisements for all Services tagged tag:k8s. Adapt its scope to your policy. LGTM leaves tags to the operator’s configured defaults. If you reuse a ProxyGroup with different device tags, use those device tags on the right-hand side of the advertisement rule. Existing access rules for proxy devices alone do not establish access to HA Services. See Tailscale’s ingress permissions.

2. Choose the proxy configuration

Use tailnetProxyClass if your proxies require custom node selection, tolerations, or security settings. The generated class does not inherit your operator’s default class. Configure replica spreading in your class and provide at least two eligible nodes for node-failure resilience. An existing ProxyGroup must be in the same cluster with spec.type: ingress. A Kubernetes API or egress group cannot be used. When tailnetProxyGroup is set, tailnetProxyClass is ignored; the group owner controls its replicas, scheduling, and lifecycle.
Switching from a blueprint-created group to an external group removes the old managed group and class. Plan a maintenance window. Do not set tailnetProxyGroup to the blueprint-created group’s name to try to transfer ownership.

3. Update the LGTM installation

In the installation’s Inputs, enable Expose to Tailnet, enter the tailnet domain and a unique hostname prefix, and select your proxy configuration. For the CLI, save this as lgtm-tailnet.patch.yaml. Replace the domain and prefix. This example creates a group with the generated class.
To use your class, set tailnetProxyClass to its name. To reuse a group, set tailnetProxyGroup to its name instead. Then apply the patch to your existing dedicated installation:
The hostname prefix must contain at most 53 lowercase letters, digits, or hyphens, starting and ending with a letter or digit. The three resulting Service names must be unused on your tailnet. Keep the installation’s existing domain input; private collector endpoints derive from the environment’s internal domain.

Schedule LGTM on a dedicated node pool

On AWS, Azure, and GCP, set Node Group (nodeGroup) and Pod Tolerations (tolerations) to match your environment’s node pool. Pod Tolerations accepts a YAML list. For a pool labeled ryvn.app/node-group-name: observability and tainted dedicated=observability:NoSchedule, add these entries to the installation inputs:
These inputs apply to Loki, Mimir, their caches, Grafana, the auth-proxy, the Grafana bootstrap job, and the managed Grafana database’s user helper. They do not create the node pool. Loki’s default ingester placement requires at least three schedulable nodes in that pool. Tailscale proxies use their ProxyClass’s scheduling settings separately. Blank inputs retain the chart defaults. If you previously edited child installations directly, first make both the LGTM and Postgres blueprint releases with these inputs available in the installation’s release channel. Set the inputs, compare the proposed child configuration with the running configuration, and preserve any other overrides before restoring blueprint management of excluded children. Removing an exclusion allows the blueprint to replace that child’s configuration.

4. Verify access and login

After the installation updates, use its grafanaUrl, lokiTailnetUrl, and mimirTailnetUrl outputs. For the example above, Grafana is https://prod-lgtm-grafana.example.ts.net. From an authorized device on the tailnet:
  1. Open Grafana and complete sign-in with your Ryvn account. Ryvn updates Grafana’s canonical URL and registers /login/generic_oauth on that hostname with its managed OIDC application. You do not need to register a separate application. Your browser must also reach your Ryvn identity provider.
  2. Query Loki and Mimir with the installation’s existing API credentials. Tailnet access does not replace their tenant authentication.
  3. Confirm new logs and metrics still arrive through the private collector path.
Have your cluster administrator confirm the three Ingresses have HTTPS addresses and the ingress ProxyGroup’s replicas are ready. For high availability, confirm placement on separate nodes and test access during a proxy restart.

Troubleshooting