Track SaaS Imports in Grafana Cloud: API Guide

This guide shows you how to track silent SaaS imports (like CSV or JSON data you push from external tools) in Grafana Cloud, either by using the built-in metrics or by building a custom API-based tracker. You’ll learn how to monitor import success, volume, and latency, and how to set up alerts so silent failures don’t go unnoticed.

What You’ll Need

• A Grafana Cloud account (free tier works for testing).
• An API key or access policy token with permissions to write metrics (for the API method).
• Access to the tool or script that performs the SaaS import (e.g., a cron job, a webhook, or a data pipeline).
• Basic familiarity with curl or a programming language like Python for API calls.

Step-by-Step

Method 1: Using Grafana Cloud’s Built-in Metrics

1. Identify the import source. Check if your SaaS tool (like a CRM, billing system, or analytics platform) already sends metrics to Grafana Cloud via a built-in integration. Look in Grafana Cloud’s “Integrations” section for your SaaS provider.
2. Enable the integration. If available, click the integration and follow the setup instructions. This usually involves entering an API key from the SaaS tool or installing a small agent.
3. Find the relevant metrics. Once data flows, explore the integration’s dashboard. Look for metrics that indicate import success (e.g., import_success_total, sync_completed) or failure (import_failure_total). If no such metric exists, you’ll need to use Method 2.
4. Set up an alert. Go to Alerting → Alert rules → New alert rule. Create a rule that triggers when the failure metric increases or when the success metric doesn’t increase within a expected time window (e.g., 24 hours).

Method 2: Custom API-Based Tracking

1. Generate an API key. In Grafana Cloud, go to Administration → Service accounts → Create a service account. Assign it a role that allows writing metrics (e.g., “Metrics Publisher”). Then create a token and copy it.
2. Get your Metrics endpoint URL. In the same menu, find your Prometheus endpoint. It looks like https://prometheus-prod-XX.grafana.net/api/prom/push.
3. Instrument your import script. In the script that runs the import (e.g., a Python script or bash script), add code to send a metric after each import attempt. Use the Prometheus remote-write protocol. Example with curl:

   curl -X POST "https://prometheus-prod-XX.grafana.net/api/prom/push" \
     -H "Authorization: Bearer <YOUR_TOKEN>" \
     --data-binary @- <
       Replace saas_import_success_total with a meaningful name. Send 1 on success, 0 on failure.

4. Add a timestamp. To track latency, send a separate metric with the duration. For example:

   saas_import_duration_seconds{source="my_saas"} 12.5

   Use the actual time the import took.

5. Send a heartbeat metric. To detect silent failures (where the script never runs), send a metric that increments every time the import runs, regardless of success:

   saas_import_runs_total{source="my_saas"} 1

6. Create a dashboard. In Grafana Cloud, create a new dashboard and add panels that query these metrics. For example, use sum(rate(saas_import_success_total[1h])) to see success rate over time.
7. Set up alerts. Go to Alerting → Alert rules → New alert rule. Create rules for:
   • Failure rate: increase(saas_import_failure_total[1h]) > 0
   • No runs: increase(saas_import_runs_total[24h]) == 0 (triggers if no import ran in 24h)
   • Latency: histogram_quantile(0.95, sum(rate(saas_import_duration_seconds_bucket[5m])) by (le)) > 60 (if you use a histogram)

Common Mistakes

• Not sending a failure metric. If you only send a success metric, you can't alert on failures. Always send both success and failure counters.
• Forgetting to handle the script crash. If the script crashes before sending any metric, you'll have no data. That's why the heartbeat metric is essential.
• Using the wrong endpoint. Make sure you use the /api/prom/push endpoint, not the query endpoint.
• Not setting a time range in alerts. Alerts need a time range (e.g., 5m) to evaluate. Without it, they may fire incorrectly.
• Ignoring the bucket for histograms. If you use a histogram for latency, you must define buckets in the script (e.g., _bucket{le="0.1"}, _bucket{le="1"}, etc.).

Tips

• Use labels to distinguish sources. Add labels like source, environment, or region to your metrics so you can filter and aggregate.
• Start with a simple counter. For the first week, just send a counter of runs and successes. Once you trust the pipeline, add latency.
• Test the API with curl first. Before modifying your script, send a test metric with curl and verify it appears in Grafana Cloud's Explore.
• Use a service account with least privilege. Only give the token the "Metrics Publisher" role, not admin.
• Set up a separate alert for no data. Use Grafana's "No data" alert condition to trigger if the metric disappears entirely.

Troubleshooting

• Metric not appearing in Explore. Check the import response. If you get a 401, your token is invalid. If 404, the endpoint is wrong. If 200, wait a few seconds and refresh Explore.
• Alert not firing. Verify the query returns data in Explore. Check the alert's evaluation interval and ensure the time range is appropriate. Also, check if the alert's folder and notification policy are set correctly.
• Duplicate metrics. If you see duplicate series, your script might be sending multiple times. Add a unique label like instance or use a counter that increments.
• High latency in alerts. If latency alerts fire too often, increase the threshold or use a longer time range (e.g., 1h instead of 5m).
• Script fails silently. Add error handling in your script to send a failure metric even if the import throws an exception. Use a try-except block in Python and send the metric in the except clause.

Frequently Asked Questions

Do I need to use the API method if my SaaS has a built-in integration?

No. If the integration provides import metrics, use it. But many integrations only show system health, not import success. In that case, the API method is better.

Can I use a webhook instead of a script?

Yes, if your SaaS can send a webhook on import completion, you can have a small serverless function (like AWS Lambda) that receives the webhook and pushes the metric to Grafana Cloud.

What if my import runs irregularly?

Use the heartbeat metric with a longer alert window (e.g., 7 days). Set the alert to trigger if increase(saas_import_runs_total[7d]) == 0.

How do I calculate success rate?

Use sum(rate(saas_import_success_total[1h])) / sum(rate(saas_import_runs_total[1h])) in a panel.

Is there a limit to how many metrics I can send?

Grafana Cloud free tier has a limit of 10,000 series. As long as you don't add high-cardinality labels (like unique IDs), you'll be fine.

Can I track imports from multiple SaaS tools?

Yes, just use a different label value for each tool, e.g., source="salesforce" and source="hubspot".

Related

• SD Card Not Showing on Mac? Fixes That Work
• Budgie vs KDE Plasma: Which Desktop Offers More?
• Unauthorized Automated Agent Use: What to Do