Merge branch 'main' into nv/schema-changes

deploy/MIGRATION.md

@@ -1,119 +0,0 @@
-# Migrating from Docker Compose to Foundry
-
-This guide walks you through migrating an existing SigNoz deployment running via
-Docker Compose to [Foundry](https://signoz.io/docs/install/docker/).
-
-## Overview
-SigNoz has developed a CLI to make installation and deployment much easier. Foundry is the evolution of how SigNoz should be installed and managed going forward.
-
-The install script is now deprecated and will no longer receive updates.
-
-To stay up to date on new installation platforms and patterns, please refer to [Foundry](https://github.com/SigNoz/foundry).
-
-Two `foundryctl` commands are used throughout this guide:
-- **`forge`** — generates deployment manifests from your `casting.yaml`. It does not touch running containers, so it is safe to re-run while you iterate.
-- **`cast`** — applies the generated manifests: it creates and starts the containers (and pulls new images).
-
-## Prerequisites
-- [ ] Install Foundry - `curl -fsSL https://signoz.io/foundry.sh | bash`
-
-## Migration Steps
-> [!WARNING]
-> **Before proceeding, back up both:**
-> - **Your docker volumes** — these hold your data.
-> - **Your existing `docker-compose.yaml` (and any config it references)** — keep a copy somewhere safe. The compose manifests are no longer distributed by SigNoz, so this backup is your only way to roll back to your previous setup.
-
-1. Make a note of the volume names used by your existing deployment for the following components:
-- ClickHouse
-- SigNoz
-- ZooKeeper
-
-> If you used the docker compose file we provided, the volumes will be `signoz-clickhouse`, `signoz-sqlite`, and `signoz-zookeeper-1`.
-
-2. Generate your `casting.yaml`. Based on internal testing, the following casting should generate the manifests that mimic the legacy docker compose setup (compare against your backed-up `docker-compose.yaml`). Once created, run `foundryctl forge -f casting.yaml`.
-
-> [!NOTE]
-> The casting contains `patches` and `config` overrides to ensure that the newly generated manifests use the existing volumes and configurations.
-
-> [!WARNING]
-> If your deployment had more than 1 shard or replica, you will need to adjust your manifest volumes accordingly. Additionally, if you had specific container images, you need to include them in your casting.
-
-> [!IMPORTANT]
-> The `replica` and `shard` macros below are placeholders. Replace them with the values from your existing ClickHouse configuration (check the `macros` section of your current ClickHouse config, e.g. `config.xml`/`metrika.xml`), otherwise the generated manifests will not match your existing data.
-
-```yaml
-apiVersion: v1alpha1
-kind: Installation
-metadata:
-  name: signoz
-spec:
-  deployment:
-    flavor: compose
-    mode: docker
-  metastore:
-    kind: sqlite
-  telemetrykeeper:
-    kind: zookeeper
-  telemetrystore:
-    spec:
-      config:
-        data:
-          config-0-0.yaml: |
-            macros:
-              replica: "example01-01-1" # replace with your existing ClickHouse replica macro (see legacy configuration files for reference)
-              shard: "01"               # replace with your existing ClickHouse shard macro (see legacy configuration files for reference)
-  patches:
-    - target: "deployment/compose.yaml"
-      operations:
-        - op: replace
-          path: /volumes/dev-telemetrykeeper-0-data/name
-          value: signoz-zookeeper-1
-        - op: replace
-          path: /volumes/dev-telemetrystore-0-0-data/name
-          value: signoz-clickhouse
-        - op: replace
-          path: /volumes/dev-metastore-sqlite-0-data/name
-          value: signoz-sqlite
-        - op: add
-          path: /services/dev-telemetrykeeper-zookeeper-0/user
-          value: root
-```
-
-> [!NOTE]
-> The `user: root` patch on the ZooKeeper service is required so the container can read/write the data in your reused ZooKeeper volume, which was created with `root`-owned files by the legacy compose setup. Without it, ZooKeeper may fail to start with permission errors.
-
-3. Validate the manifests in `pours/deployment`. Pay special attention to `compose.yaml` — it should mimic the legacy manifest and the configuration files needed for `clickhouse`. **Do note that these are now in YAML instead of XML.**
-
-   If you had custom settings for features like SMTP or ingestion processors/receivers, you will need to include those in your casting file.
-
-4. Execute a `docker compose down`. **Do not** include parameters such as `--volumes` (or `-v`), as it will wipe the volumes we need to maintain and reuse to avoid data loss. **NOTE: this will generate downtime so please plan accordingly**.
-
-5. Validate the SigNoz containers are down with `docker ps -a`. Multiple containers cannot bind the same volume.
-
-6. Run `foundryctl cast -f casting.yaml`. This will recreate the containers based on the spec. This process will download new container images.
-
-> [!NOTE]
-> When `cast` is run, the migration container will execute its migrations.
-
-## Verifying the Migration
-- SigNoz containers will be up and running.
-- Log in to the SigNoz UI and verify that data is present.
-- Validate that your data ingestion is receiving data.
-- Review the logs from both ClickHouse and ZooKeeper; no errors should be present.
-
-## Rolling Back
-Because step 4 brought the legacy stack down *without* `-v`, your original volumes
-are untouched and still hold your data. To roll back:
-
-- Stop and remove the containers created by Foundry (`docker compose down`, again without `-v`).
-- Confirm the containers are gone with `docker ps -a` so nothing else is bound to the volumes.
-- Reapply your original docker compose file (`docker compose up -d`). It will reattach to the
-  existing volumes and restore your prior state.
-
-## Troubleshooting
-- Please reach out to our community on [Slack](https://signoz.io/slack).
-
-## References
-- [SigNoz Docker installation docs](https://signoz.io/docs/install/docker/)
-- [SigNoz documentation](https://signoz.io/docs)
-- [Foundry](https://github.com/SigNoz/foundry)

deploy/README.md

@@ -3,18 +3,77 @@
 Check that you have cloned [signoz/signoz](https://github.com/signoz/signoz)
 and currently are in `signoz/deploy` folder.
 
-## Installation
+## Docker
 
-> **Note:** The `install.sh` script and the `docker-compose` manifests have been deprecated.
+If you don't have docker set up, please follow [this guide](https://docs.docker.com/engine/install/)
+to set up docker before proceeding with the next steps.
 
-SigNoz now installs and runs through [Foundry](https://signoz.io/docs/install/docker/).
+### Using Install Script
 
-> **Already running SigNoz via Docker Compose?** See the [Migration Guide](./MIGRATION.md) to transition your existing deployment to Foundry.
+Now run the following command to install:
 
-Please follow the latest installation instructions at [signoz.io/docs/install/docker](https://signoz.io/docs/install/docker/).
-Foundry has support for **different platforms and architectures**, please review the project documentation for more details.
+```sh
+./install.sh
+```
 
-> You will need Docker installed first. If you don't have it set up, follow [this guide](https://docs.docker.com/engine/install/).
+### Using Docker Compose
+
+If you don't have docker compose set up, please follow [this guide](https://docs.docker.com/compose/install/)
+to set up docker compose before proceeding with the next steps.
+
+```sh
+cd deploy/docker
+docker compose up -d
+```
+
+Open http://localhost:8080 in your favourite browser.
+
+To start collecting logs and metrics from your infrastructure, run the following command:
+
+```sh
+cd generator/infra
+docker compose up -d
+```
+
+To start generating sample traces, run the following command:
+
+```sh
+cd generator/hotrod
+docker compose up -d
+```
+
+In a couple of minutes, you should see the data generated from hotrod in SigNoz UI.
+
+For more details, please refer to the [SigNoz documentation](https://signoz.io/docs/install/docker/).
+
+## Docker Swarm
+
+To install SigNoz using Docker Swarm, run the following command:
+
+```sh
+cd deploy/docker-swarm
+docker stack deploy -c docker-compose.yaml signoz
+```
+
+Open http://localhost:8080 in your favourite browser.
+
+To start collecting logs and metrics from your infrastructure, run the following command:
+
+```sh
+cd generator/infra
+docker stack deploy -c docker-compose.yaml infra
+```
+
+To start generating sample traces, run the following command:
+
+```sh
+cd generator/hotrod
+docker stack deploy -c docker-compose.yaml hotrod
+```
+
+In a couple of minutes, you should see the data generated from hotrod in SigNoz UI.
+
+For more details, please refer to the [SigNoz documentation](https://signoz.io/docs/install/docker-swarm/).
 
 ## Uninstall/Troubleshoot?
 

deploy/common/clickhouse/cluster.ha.xml

@@ -0,0 +1,75 @@
+<?xml version="1.0"?>
+<clickhouse>
+    <!-- ZooKeeper is used to store metadata about replicas, when using Replicated tables.
+         Optional. If you don't use replicated tables, you could omit that.
+
+         See https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replication/
+      -->
+    <zookeeper>
+        <node index="1">
+            <host>zookeeper-1</host>
+            <port>2181</port>
+        </node>
+        <node index="2">
+            <host>zookeeper-2</host>
+            <port>2181</port>
+        </node>
+        <node index="3">
+            <host>zookeeper-3</host>
+            <port>2181</port>
+        </node>
+    </zookeeper>
+
+    <!-- Configuration of clusters that could be used in Distributed tables.
+         https://clickhouse.com/docs/en/operations/table_engines/distributed/
+      -->
+    <remote_servers>
+        <cluster>
+            <!-- Inter-server per-cluster secret for Distributed queries
+                 default: no secret (no authentication will be performed)
+
+                 If set, then Distributed queries will be validated on shards, so at least:
+                 - such cluster should exist on the shard,
+                 - such cluster should have the same secret.
+
+                 And also (and which is more important), the initial_user will
+                 be used as current user for the query.
+
+                 Right now the protocol is pretty simple and it only takes into account:
+                 - cluster name
+                 - query
+
+                 Also it will be nice if the following will be implemented:
+                 - source hostname (see interserver_http_host), but then it will depends from DNS,
+                   it can use IP address instead, but then the you need to get correct on the initiator node.
+                 - target hostname / ip address (same notes as for source hostname)
+                 - time-based security tokens
+            -->
+            <!-- <secret></secret> -->
+            <shard>
+                <!-- Optional. Whether to write data to just one of the replicas. Default: false (write data to all replicas). -->
+                <!-- <internal_replication>false</internal_replication> -->
+                <!-- Optional. Shard weight when writing data. Default: 1. -->
+                <!-- <weight>1</weight> -->
+                <replica>
+                    <host>clickhouse</host>
+                    <port>9000</port>
+                    <!-- Optional. Priority of the replica for load_balancing. Default: 1 (less value has more priority). -->
+                    <!-- <priority>1</priority> -->
+                </replica>
+            </shard>
+            <shard>
+                <replica>
+                    <host>clickhouse-2</host>
+                    <port>9000</port>
+                </replica>
+            </shard>
+            <shard>
+                <replica>
+                    <host>clickhouse-3</host>
+                    <port>9000</port>
+                </replica>
+            </shard>
+        </cluster>
+    </remote_servers>
+</clickhouse>

deploy/common/clickhouse/cluster.xml

@@ -0,0 +1,75 @@
+<?xml version="1.0"?>
+<clickhouse>
+    <!-- ZooKeeper is used to store metadata about replicas, when using Replicated tables.
+         Optional. If you don't use replicated tables, you could omit that.
+
+         See https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replication/
+      -->
+    <zookeeper>
+        <node index="1">
+            <host>zookeeper-1</host>
+            <port>2181</port>
+        </node>
+        <!-- <node index="2">
+            <host>zookeeper-2</host>
+            <port>2181</port>
+        </node>
+        <node index="3">
+            <host>zookeeper-3</host>
+            <port>2181</port>
+        </node> -->
+    </zookeeper>
+
+    <!-- Configuration of clusters that could be used in Distributed tables.
+         https://clickhouse.com/docs/en/operations/table_engines/distributed/
+      -->
+    <remote_servers>
+        <cluster>
+            <!-- Inter-server per-cluster secret for Distributed queries
+                 default: no secret (no authentication will be performed)
+
+                 If set, then Distributed queries will be validated on shards, so at least:
+                 - such cluster should exist on the shard,
+                 - such cluster should have the same secret.
+
+                 And also (and which is more important), the initial_user will
+                 be used as current user for the query.
+
+                 Right now the protocol is pretty simple and it only takes into account:
+                 - cluster name
+                 - query
+
+                 Also it will be nice if the following will be implemented:
+                 - source hostname (see interserver_http_host), but then it will depends from DNS,
+                   it can use IP address instead, but then the you need to get correct on the initiator node.
+                 - target hostname / ip address (same notes as for source hostname)
+                 - time-based security tokens
+            -->
+            <!-- <secret></secret> -->
+            <shard>
+                <!-- Optional. Whether to write data to just one of the replicas. Default: false (write data to all replicas). -->
+                <!-- <internal_replication>false</internal_replication> -->
+                <!-- Optional. Shard weight when writing data. Default: 1. -->
+                <!-- <weight>1</weight> -->
+                <replica>
+                    <host>clickhouse</host>
+                    <port>9000</port>
+                    <!-- Optional. Priority of the replica for load_balancing. Default: 1 (less value has more priority). -->
+                    <!-- <priority>1</priority> -->
+                </replica>
+            </shard>
+            <!-- <shard>
+                <replica>
+                    <host>clickhouse-2</host>
+                    <port>9000</port>
+                </replica>
+            </shard>
+            <shard>
+                <replica>
+                    <host>clickhouse-3</host>
+                    <port>9000</port>
+                </replica>
+            </shard> -->
+        </cluster>
+    </remote_servers>
+</clickhouse>

deploy/common/clickhouse/custom-function.xml

@@ -0,0 +1,21 @@
+<functions>
+    <function>
+        <type>executable</type>
+        <name>histogramQuantile</name>
+        <return_type>Float64</return_type>
+        <argument>
+            <type>Array(Float64)</type>
+            <name>buckets</name>
+        </argument>
+        <argument>
+            <type>Array(Float64)</type>
+            <name>counts</name>
+        </argument>
+        <argument>
+            <type>Float64</type>
+            <name>quantile</name>
+        </argument>
+        <format>CSV</format>
+        <command>./histogramQuantile</command>
+    </function>
+</functions>

deploy/common/clickhouse/storage.xml

@@ -0,0 +1,41 @@
+<?xml version="1.0"?>
+<clickhouse>
+<storage_configuration>
+    <disks>
+        <default>
+            <keep_free_space_bytes>10485760</keep_free_space_bytes>
+        </default>
+        <s3>
+            <type>s3</type>
+            <!-- For S3 cold storage,
+                    if region is us-east-1, endpoint can be https://<bucket-name>.s3.amazonaws.com
+                    if region is not us-east-1, endpoint should be https://<bucket-name>.s3-<region>.amazonaws.com
+                For GCS cold storage,
+                    endpoint should be https://storage.googleapis.com/<bucket-name>/data/
+                -->
+            <endpoint>https://BUCKET-NAME.s3-REGION-NAME.amazonaws.com/data/</endpoint>
+            <access_key_id>ACCESS-KEY-ID</access_key_id>
+            <secret_access_key>SECRET-ACCESS-KEY</secret_access_key>
+            <!-- In case of S3, uncomment the below configuration in case you want to read
+                AWS credentials from the Environment variables if they exist. -->
+            <!-- <use_environment_credentials>true</use_environment_credentials> -->
+            <!-- In case of GCS, uncomment the below configuration, since GCS does
+                not support batch deletion and result in error messages in logs. -->
+            <!-- <support_batch_delete>false</support_batch_delete> -->
+        </s3>
+   </disks>
+   <policies>
+       <tiered>
+           <volumes>
+                <default>
+                    <disk>default</disk>
+                </default>
+                <s3>
+                    <disk>s3</disk>
+                    <perform_ttl_move_on_insert>0</perform_ttl_move_on_insert>
+                </s3>
+            </volumes>
+        </tiered>
+    </policies>
+</storage_configuration>
+</clickhouse>

deploy/common/clickhouse/users.xml

@@ -0,0 +1,123 @@
+<?xml version="1.0"?>
+<clickhouse>
+    <!-- See also the files in users.d directory where the settings can be overridden. -->
+
+    <!-- Profiles of settings. -->
+    <profiles>
+        <!-- Default settings. -->
+        <default>
+            <!-- Maximum memory usage for processing single query, in bytes. -->
+            <max_memory_usage>10000000000</max_memory_usage>
+
+            <!-- How to choose between replicas during distributed query processing.
+                 random - choose random replica from set of replicas with minimum number of errors
+                 nearest_hostname - from set of replicas with minimum number of errors, choose replica
+                  with minimum number of different symbols between replica's hostname and local hostname
+                  (Hamming distance).
+                 in_order - first live replica is chosen in specified order.
+                 first_or_random - if first replica one has higher number of errors, pick a random one from replicas with minimum number of errors.
+            -->
+            <load_balancing>random</load_balancing>
+        </default>
+
+        <!-- Profile that allows only read queries. -->
+        <readonly>
+            <readonly>1</readonly>
+        </readonly>
+    </profiles>
+
+    <!-- Users and ACL. -->
+    <users>
+        <!-- If user name was not specified, 'default' user is used. -->
+        <default>
+            <!-- See also the files in users.d directory where the password can be overridden.
+
+                 Password could be specified in plaintext or in SHA256 (in hex format).
+
+                 If you want to specify password in plaintext (not recommended), place it in 'password' element.
+                 Example: <password>qwerty</password>.
+                 Password could be empty.
+
+                 If you want to specify SHA256, place it in 'password_sha256_hex' element.
+                 Example: <password_sha256_hex>65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5</password_sha256_hex>
+                 Restrictions of SHA256: impossibility to connect to ClickHouse using MySQL JS client (as of July 2019).
+
+                 If you want to specify double SHA1, place it in 'password_double_sha1_hex' element.
+                 Example: <password_double_sha1_hex>e395796d6546b1b65db9d665cd43f0e858dd4303</password_double_sha1_hex>
+
+                 If you want to specify a previously defined LDAP server (see 'ldap_servers' in the main config) for authentication,
+                  place its name in 'server' element inside 'ldap' element.
+                 Example: <ldap><server>my_ldap_server</server></ldap>
+
+                 If you want to authenticate the user via Kerberos (assuming Kerberos is enabled, see 'kerberos' in the main config),
+                  place 'kerberos' element instead of 'password' (and similar) elements.
+                 The name part of the canonical principal name of the initiator must match the user name for authentication to succeed.
+                 You can also place 'realm' element inside 'kerberos' element to further restrict authentication to only those requests
+                  whose initiator's realm matches it.
+                 Example: <kerberos />
+                 Example: <kerberos><realm>EXAMPLE.COM</realm></kerberos>
+
+                 How to generate decent password:
+                 Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'
+                 In first line will be password and in second - corresponding SHA256.
+
+                 How to generate double SHA1:
+                 Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'
+                 In first line will be password and in second - corresponding double SHA1.
+            -->
+            <password></password>
+
+            <!-- List of networks with open access.
+
+                 To open access from everywhere, specify:
+                    <ip>::/0</ip>
+
+                 To open access only from localhost, specify:
+                    <ip>::1</ip>
+                    <ip>127.0.0.1</ip>
+
+                 Each element of list has one of the following forms:
+                 <ip> IP-address or network mask. Examples: 213.180.204.3 or 10.0.0.1/8 or 10.0.0.1/255.255.255.0
+                     2a02:6b8::3 or 2a02:6b8::3/64 or 2a02:6b8::3/ffff:ffff:ffff:ffff::.
+                 <host> Hostname. Example: server01.clickhouse.com.
+                     To check access, DNS query is performed, and all received addresses compared to peer address.
+                 <host_regexp> Regular expression for host names. Example, ^server\d\d-\d\d-\d\.clickhouse\.com$
+                     To check access, DNS PTR query is performed for peer address and then regexp is applied.
+                     Then, for result of PTR query, another DNS query is performed and all received addresses compared to peer address.
+                     Strongly recommended that regexp is ends with $
+                 All results of DNS requests are cached till server restart.
+            -->
+            <networks>
+                <ip>::/0</ip>
+            </networks>
+
+            <!-- Settings profile for user. -->
+            <profile>default</profile>
+
+            <!-- Quota for user. -->
+            <quota>default</quota>
+
+            <!-- User can create other users and grant rights to them. -->
+            <!-- <access_management>1</access_management> -->
+        </default>
+    </users>
+
+    <!-- Quotas. -->
+    <quotas>
+        <!-- Name of quota. -->
+        <default>
+            <!-- Limits for time interval. You could specify many intervals with different limits. -->
+            <interval>
+                <!-- Length of interval. -->
+                <duration>3600</duration>
+
+                <!-- No limits. Just calculate resource usage for time interval. -->
+                <queries>0</queries>
+                <errors>0</errors>
+                <result_rows>0</result_rows>
+                <read_rows>0</read_rows>
+                <execution_time>0</execution_time>
+            </interval>
+        </default>
+    </quotas>
+</clickhouse>

deploy/common/locust-scripts/locustfile.py

@@ -0,0 +1,16 @@
+from locust import HttpUser, task, between
+class UserTasks(HttpUser):
+    wait_time = between(5, 15)
+
+    @task
+    def rachel(self):
+        self.client.get("/dispatch?customer=123&nonse=0.6308392664170006")
+    @task
+    def trom(self):
+        self.client.get("/dispatch?customer=392&nonse=0.015296363321630757")
+    @task
+    def japanese(self):
+        self.client.get("/dispatch?customer=731&nonse=0.8022286220408668")
+    @task
+    def coffee(self):
+        self.client.get("/dispatch?customer=567&nonse=0.0022220379420636593")

deploy/common/signoz/otel-collector-opamp-config.yaml

@@ -0,0 +1 @@
+server_endpoint: ws://signoz:4320/v1/opamp

deploy/common/signoz/prometheus.yml

@@ -0,0 +1,25 @@
+# my global config
+global:
+  scrape_interval:     5s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
+  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
+  # scrape_timeout is set to the global default (10s).
+
+# Alertmanager configuration
+alerting:
+  alertmanagers:
+  - static_configs:
+    - targets:
+      - alertmanager:9093
+
+# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
+rule_files: []
+  # - "first_rules.yml"
+  # - "second_rules.yml"
+  # - 'alerts.yml'
+
+# A scrape configuration containing exactly one endpoint to scrape:
+# Here it's Prometheus itself.
+scrape_configs: []
+
+remote_read:
+  - url: tcp://clickhouse:9000/signoz_metrics

deploy/docker-swarm/docker-compose.ha.yaml

@@ -0,0 +1,288 @@
+version: "3"
+x-common: &common
+  networks:
+    - signoz-net
+  deploy:
+    restart_policy:
+      condition: on-failure
+  logging:
+    options:
+      max-size: 50m
+      max-file: "3"
+x-clickhouse-defaults: &clickhouse-defaults
+  !!merge <<: *common
+  image: clickhouse/clickhouse-server:25.5.6
+  tty: true
+  deploy:
+    labels:
+      signoz.io/scrape: "true"
+      signoz.io/port: "9363"
+      signoz.io/path: "/metrics"
+  depends_on:
+    - zookeeper-1
+    - zookeeper-2
+    - zookeeper-3
+  healthcheck:
+    test:
+      - CMD
+      - wget
+      - --spider
+      - -q
+      - 0.0.0.0:8123/ping
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  ulimits:
+    nproc: 65535
+    nofile:
+      soft: 262144
+      hard: 262144
+  environment:
+    - CLICKHOUSE_SKIP_USER_SETUP=1
+x-zookeeper-defaults: &zookeeper-defaults
+  !!merge <<: *common
+  image: signoz/zookeeper:3.7.1
+  user: root
+  deploy:
+    labels:
+      signoz.io/scrape: "true"
+      signoz.io/port: "9141"
+      signoz.io/path: "/metrics"
+  healthcheck:
+    test:
+      - CMD-SHELL
+      - curl -s -m 2 http://localhost:8080/commands/ruok | grep error | grep null
+    interval: 30s
+    timeout: 5s
+    retries: 3
+x-db-depend: &db-depend
+  !!merge <<: *common
+  depends_on:
+    - clickhouse
+    - clickhouse-2
+    - clickhouse-3
+services:
+  init-clickhouse:
+    !!merge <<: *common
+    image: clickhouse/clickhouse-server:25.5.6
+    command:
+      - bash
+      - -c
+      - |
+        version="v0.0.1"
+        node_os=$$(uname -s | tr '[:upper:]' '[:lower:]')
+        node_arch=$$(uname -m | sed s/aarch64/arm64/ | sed s/x86_64/amd64/)
+        echo "Fetching histogram-binary for $${node_os}/$${node_arch}"
+        cd /tmp
+        wget -O histogram-quantile.tar.gz "https://github.com/SigNoz/signoz/releases/download/histogram-quantile%2F$${version}/histogram-quantile_$${node_os}_$${node_arch}.tar.gz"
+        tar -xvzf histogram-quantile.tar.gz
+        mv histogram-quantile /var/lib/clickhouse/user_scripts/histogramQuantile
+    deploy:
+      restart_policy:
+        condition: on-failure
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+  zookeeper-1:
+    !!merge <<: *zookeeper-defaults
+    # ports:
+    #   - "2181:2181"
+    #   - "2888:2888"
+    #   - "3888:3888"
+    volumes:
+      - ./clickhouse-setup/data/zookeeper-1:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=1
+      - ZOO_SERVERS=0.0.0.0:2888:3888,zookeeper-2:2888:3888,zookeeper-3:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  zookeeper-2:
+    !!merge <<: *zookeeper-defaults
+    # ports:
+    #   - "2182:2181"
+    #   - "2889:2888"
+    #   - "3889:3888"
+    volumes:
+      - ./clickhouse-setup/data/zookeeper-2:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=2
+      - ZOO_SERVERS=zookeeper-1:2888:3888,0.0.0.0:2888:3888,zookeeper-3:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  zookeeper-3:
+    !!merge <<: *zookeeper-defaults
+    # ports:
+    #   - "2183:2181"
+    #   - "2890:2888"
+    #   - "3890:3888"
+    volumes:
+      - ./clickhouse-setup/data/zookeeper-3:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=3
+      - ZOO_SERVERS=zookeeper-1:2888:3888,zookeeper-2:2888:3888,0.0.0.0:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  clickhouse:
+    !!merge <<: *clickhouse-defaults
+    # TODO: needed for schema-migrator to work, remove this redundancy once we have a better solution
+    hostname: clickhouse
+    # ports:
+    #   - "9000:9000"
+    #   - "8123:8123"
+    #   - "9181:9181"
+    configs:
+      - source: clickhouse-config
+        target: /etc/clickhouse-server/config.xml
+      - source: clickhouse-users
+        target: /etc/clickhouse-server/users.xml
+      - source: clickhouse-custom-function
+        target: /etc/clickhouse-server/custom-function.xml
+      - source: clickhouse-cluster
+        target: /etc/clickhouse-server/config.d/cluster.ha.xml
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ./clickhouse-setup/data/clickhouse/:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  clickhouse-2:
+    !!merge <<: *clickhouse-defaults
+    hostname: clickhouse-2
+    # ports:
+    #   - "9001:9000"
+    #   - "8124:8123"
+    #   - "9182:9181"
+    configs:
+      - source: clickhouse-config
+        target: /etc/clickhouse-server/config.xml
+      - source: clickhouse-users
+        target: /etc/clickhouse-server/users.xml
+      - source: clickhouse-custom-function
+        target: /etc/clickhouse-server/custom-function.xml
+      - source: clickhouse-cluster
+        target: /etc/clickhouse-server/config.d/cluster.ha.xml
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ./clickhouse-setup/data/clickhouse-2/:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  clickhouse-3:
+    !!merge <<: *clickhouse-defaults
+    hostname: clickhouse-3
+    # ports:
+    #   - "9002:9000"
+    #   - "8125:8123"
+    #   - "9183:9181"
+    configs:
+      - source: clickhouse-config
+        target: /etc/clickhouse-server/config.xml
+      - source: clickhouse-users
+        target: /etc/clickhouse-server/users.xml
+      - source: clickhouse-custom-function
+        target: /etc/clickhouse-server/custom-function.xml
+      - source: clickhouse-cluster
+        target: /etc/clickhouse-server/config.d/cluster.ha.xml
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ./clickhouse-setup/data/clickhouse-3/:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  signoz:
+    !!merge <<: *db-depend
+    image: signoz/signoz:v0.129.0
+    ports:
+      - "8080:8080" # signoz port
+    #   - "6060:6060"     # pprof port
+    volumes:
+      - ./clickhouse-setup/data/signoz/:/var/lib/signoz/
+    environment:
+      - SIGNOZ_ALERTMANAGER_PROVIDER=signoz
+      - SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
+      - SIGNOZ_TOKENIZER_JWT_SECRET=secret
+    healthcheck:
+      test:
+        - CMD
+        - wget
+        - --spider
+        - -q
+        - localhost:8080/api/v1/health
+      interval: 30s
+      timeout: 5s
+      retries: 3
+  otel-collector:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:v0.144.5
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate sync check &&
+        /signoz-otel-collector --config=/etc/otel-collector-config.yaml --manager-config=/etc/manager-config.yaml --copy-path=/var/tmp/collector-config.yaml
+    configs:
+      - source: otel-collector-config
+        target: /etc/otel-collector-config.yaml
+      - source: otel-manager-config
+        target: /etc/manager-config.yaml
+    environment:
+      - OTEL_RESOURCE_ATTRIBUTES=host.name={{.Node.Hostname}},os.type={{.Node.Platform.OS}}
+      - LOW_CARDINAL_EXCEPTION_GROUPING=false
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    ports:
+      # - "1777:1777"     # pprof extension
+      - "4317:4317" # OTLP gRPC receiver
+      - "4318:4318" # OTLP HTTP receiver
+    deploy:
+      replicas: 3
+  signoz-telemetrystore-migrator:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:v0.144.5
+    environment:
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate bootstrap &&
+        /signoz-otel-collector migrate sync up &&
+        /signoz-otel-collector migrate async up
+networks:
+  signoz-net:
+    name: signoz-net
+volumes:
+  clickhouse:
+    name: signoz-clickhouse
+  clickhouse-2:
+    name: signoz-clickhouse-2
+  clickhouse-3:
+    name: signoz-clickhouse-3
+  sqlite:
+    name: signoz-sqlite
+  zookeeper-1:
+    name: signoz-zookeeper-1
+  zookeeper-2:
+    name: signoz-zookeeper-2
+  zookeeper-3:
+    name: signoz-zookeeper-3
+configs:
+  clickhouse-config:
+    file: ../common/clickhouse/config.xml
+  clickhouse-users:
+    file: ../common/clickhouse/users.xml
+  clickhouse-custom-function:
+    file: ../common/clickhouse/custom-function.xml
+  clickhouse-cluster:
+    file: ../common/clickhouse/cluster.ha.xml
+  otel-collector-config:
+    file: ./otel-collector-config.yaml
+  otel-manager-config:
+    file: ../common/signoz/otel-collector-opamp-config.yaml

deploy/docker-swarm/docker-compose.yaml

@@ -0,0 +1,206 @@
+version: "3"
+x-common: &common
+  networks:
+    - signoz-net
+  deploy:
+    restart_policy:
+      condition: on-failure
+  logging:
+    options:
+      max-size: 50m
+      max-file: "3"
+x-clickhouse-defaults: &clickhouse-defaults
+  !!merge <<: *common
+  image: clickhouse/clickhouse-server:25.5.6
+  tty: true
+  deploy:
+    labels:
+      signoz.io/scrape: "true"
+      signoz.io/port: "9363"
+      signoz.io/path: "/metrics"
+  depends_on:
+    - init-clickhouse
+    - zookeeper-1
+  healthcheck:
+    test:
+      - CMD
+      - wget
+      - --spider
+      - -q
+      - 0.0.0.0:8123/ping
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  ulimits:
+    nproc: 65535
+    nofile:
+      soft: 262144
+      hard: 262144
+  environment:
+    - CLICKHOUSE_SKIP_USER_SETUP=1
+x-zookeeper-defaults: &zookeeper-defaults
+  !!merge <<: *common
+  image: signoz/zookeeper:3.7.1
+  user: root
+  deploy:
+    labels:
+      signoz.io/scrape: "true"
+      signoz.io/port: "9141"
+      signoz.io/path: "/metrics"
+  healthcheck:
+    test:
+      - CMD-SHELL
+      - curl -s -m 2 http://localhost:8080/commands/ruok | grep error | grep null
+    interval: 30s
+    timeout: 5s
+    retries: 3
+x-db-depend: &db-depend
+  !!merge <<: *common
+  depends_on:
+    - clickhouse
+services:
+  init-clickhouse:
+    !!merge <<: *common
+    image: clickhouse/clickhouse-server:25.5.6
+    command:
+      - bash
+      - -c
+      - |
+        version="v0.0.1"
+        node_os=$$(uname -s | tr '[:upper:]' '[:lower:]')
+        node_arch=$$(uname -m | sed s/aarch64/arm64/ | sed s/x86_64/amd64/)
+        echo "Fetching histogram-binary for $${node_os}/$${node_arch}"
+        cd /tmp
+        wget -O histogram-quantile.tar.gz "https://github.com/SigNoz/signoz/releases/download/histogram-quantile%2F$${version}/histogram-quantile_$${node_os}_$${node_arch}.tar.gz"
+        tar -xvzf histogram-quantile.tar.gz
+        mv histogram-quantile /var/lib/clickhouse/user_scripts/histogramQuantile
+    deploy:
+      restart_policy:
+        condition: on-failure
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+  zookeeper-1:
+    !!merge <<: *zookeeper-defaults
+    # ports:
+    #   - "2181:2181"
+    #   - "2888:2888"
+    #   - "3888:3888"
+    volumes:
+      - zookeeper-1:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=1
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  clickhouse:
+    !!merge <<: *clickhouse-defaults
+    # TODO: needed for clickhouse TCP connectio
+    hostname: clickhouse
+    # ports:
+    #   - "9000:9000"
+    #   - "8123:8123"
+    #   - "9181:9181"
+
+    configs:
+      - source: clickhouse-config
+        target: /etc/clickhouse-server/config.xml
+      - source: clickhouse-users
+        target: /etc/clickhouse-server/users.xml
+      - source: clickhouse-custom-function
+        target: /etc/clickhouse-server/custom-function.xml
+      - source: clickhouse-cluster
+        target: /etc/clickhouse-server/config.d/cluster.xml
+    volumes:
+      - clickhouse:/var/lib/clickhouse/
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  signoz:
+    !!merge <<: *db-depend
+    image: signoz/signoz:v0.129.0
+    ports:
+      - "8080:8080" # signoz port
+    volumes:
+      - sqlite:/var/lib/signoz/
+    environment:
+      - SIGNOZ_ALERTMANAGER_PROVIDER=signoz
+      - SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
+      - SIGNOZ_TOKENIZER_JWT_SECRET=secret
+    healthcheck:
+      test:
+        - CMD
+        - wget
+        - --spider
+        - -q
+        - localhost:8080/api/v1/health
+      interval: 30s
+      timeout: 5s
+      retries: 3
+  otel-collector:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:v0.144.5
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate sync check &&
+        /signoz-otel-collector --config=/etc/otel-collector-config.yaml --manager-config=/etc/manager-config.yaml --copy-path=/var/tmp/collector-config.yaml
+    configs:
+      - source: otel-collector-config
+        target: /etc/otel-collector-config.yaml
+      - source: otel-manager-config
+        target: /etc/manager-config.yaml
+    environment:
+      - OTEL_RESOURCE_ATTRIBUTES=host.name={{.Node.Hostname}},os.type={{.Node.Platform.OS}}
+      - LOW_CARDINAL_EXCEPTION_GROUPING=false
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    ports:
+      # - "1777:1777"     # pprof extension
+      - "4317:4317" # OTLP gRPC receiver
+      - "4318:4318" # OTLP HTTP receiver
+    deploy:
+      replicas: 3
+  signoz-telemetrystore-migrator:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:v0.144.5
+    environment:
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate bootstrap &&
+        /signoz-otel-collector migrate sync up &&
+        /signoz-otel-collector migrate async up
+networks:
+  signoz-net:
+    name: signoz-net
+volumes:
+  clickhouse:
+    name: signoz-clickhouse
+  sqlite:
+    name: signoz-sqlite
+  zookeeper-1:
+    name: signoz-zookeeper-1
+configs:
+  clickhouse-config:
+    file: ../common/clickhouse/config.xml
+  clickhouse-users:
+    file: ../common/clickhouse/users.xml
+  clickhouse-custom-function:
+    file: ../common/clickhouse/custom-function.xml
+  clickhouse-cluster:
+    file: ../common/clickhouse/cluster.xml
+  otel-collector-config:
+    file: ./otel-collector-config.yaml
+  otel-manager-config:
+    file: ../common/signoz/otel-collector-opamp-config.yaml

deploy/docker-swarm/otel-collector-config.yaml

@@ -0,0 +1,118 @@
+connectors:
+  signozmeter:
+    metrics_flush_interval: 1h
+    dimensions:
+      - name: service.name
+      - name: deployment.environment
+      - name: host.name
+receivers:
+  otlp:
+    protocols:
+      grpc:
+        endpoint: 0.0.0.0:4317
+      http:
+        endpoint: 0.0.0.0:4318
+  prometheus:
+    config:
+      global:
+        scrape_interval: 60s
+      scrape_configs:
+        - job_name: otel-collector
+          static_configs:
+          - targets:
+              - localhost:8888
+            labels:
+              job_name: otel-collector
+processors:
+  batch:
+    send_batch_size: 10000
+    send_batch_max_size: 11000
+    timeout: 10s
+  batch/meter:
+    send_batch_max_size: 25000
+    send_batch_size: 20000
+    timeout: 1s
+  resourcedetection:
+    # Using OTEL_RESOURCE_ATTRIBUTES envvar, env detector adds custom labels.
+    detectors: [env, system]
+    timeout: 2s
+  signozspanmetrics/delta:
+    metrics_exporter: signozclickhousemetrics
+    metrics_flush_interval: 60s
+    latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ]
+    dimensions_cache_size: 100000
+    aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
+    enable_exp_histogram: true
+    dimensions:
+      - name: service.namespace
+        default: default
+      - name: deployment.environment
+        default: default
+      # This is added to ensure the uniqueness of the timeseries
+      # Otherwise, identical timeseries produced by multiple replicas of
+      # collectors result in incorrect APM metrics
+      - name: signoz.collector.id
+      - name: service.version
+      - name: browser.platform
+      - name: browser.mobile
+      - name: k8s.cluster.name
+      - name: k8s.node.name
+      - name: k8s.namespace.name
+      - name: host.name
+      - name: host.type
+      - name: container.name
+extensions:
+  health_check:
+    endpoint: 0.0.0.0:13133
+  pprof:
+    endpoint: 0.0.0.0:1777
+exporters:
+  clickhousetraces:
+    datasource: tcp://clickhouse:9000/signoz_traces
+    low_cardinal_exception_grouping: ${env:LOW_CARDINAL_EXCEPTION_GROUPING}
+    use_new_schema: true
+  signozclickhousemetrics:
+    dsn: tcp://clickhouse:9000/signoz_metrics
+  clickhouselogsexporter:
+    dsn: tcp://clickhouse:9000/signoz_logs
+    timeout: 10s
+    use_new_schema: true
+  signozclickhousemeter:
+    dsn: tcp://clickhouse:9000/signoz_meter
+    timeout: 45s
+    sending_queue:
+      enabled: false
+  metadataexporter:
+    cache:
+      provider: in_memory
+    dsn: tcp://clickhouse:9000/signoz_metadata
+    enabled: true
+    timeout: 45s
+service:
+  telemetry:
+    logs:
+      encoding: json
+  extensions:
+    - health_check
+    - pprof
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: [signozspanmetrics/delta, batch]
+      exporters: [clickhousetraces, metadataexporter, signozmeter]
+    metrics:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [signozclickhousemetrics, metadataexporter, signozmeter]
+    metrics/prometheus:
+      receivers: [prometheus]
+      processors: [batch]
+      exporters: [signozclickhousemetrics, metadataexporter, signozmeter]
+    logs:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [clickhouselogsexporter, metadataexporter, signozmeter]
+    metrics/meter:
+      receivers: [signozmeter]
+      processors: [batch/meter]
+      exporters: [signozclickhousemeter]

deploy/docker/.env

@@ -0,0 +1 @@
+COMPOSE_PROJECT_NAME=signoz

deploy/docker/clickhouse-setup/.deprecated

@@ -0,0 +1,3 @@
+This data directory is deprecated and will be removed in the future.
+Please use the migration script under `scripts/volume-migration` to migrate data from bind mounts to Docker volumes.
+The script also renames the project name to `signoz` and the network name to `signoz-net` (if not already in place).

deploy/docker/clickhouse-setup/user_scripts/.deprecated

@@ -0,0 +1,2 @@
+This directory is deprecated and will be removed in the future.
+Please use the new directory for Clickhouse setup scripts: `scripts/clickhouse` instead.

deploy/docker/docker-compose.ha.yaml

@@ -0,0 +1,265 @@
+version: "3"
+x-common: &common
+  networks:
+    - signoz-net
+  restart: unless-stopped
+  logging:
+    options:
+      max-size: 50m
+      max-file: "3"
+x-clickhouse-defaults: &clickhouse-defaults
+  !!merge <<: *common
+  # addding non LTS version due to this fix https://github.com/ClickHouse/ClickHouse/commit/32caf8716352f45c1b617274c7508c86b7d1afab
+  image: clickhouse/clickhouse-server:25.5.6
+  tty: true
+  labels:
+    signoz.io/scrape: "true"
+    signoz.io/port: "9363"
+    signoz.io/path: "/metrics"
+  depends_on:
+    init-clickhouse:
+      condition: service_completed_successfully
+    zookeeper-1:
+      condition: service_healthy
+    zookeeper-2:
+      condition: service_healthy
+    zookeeper-3:
+      condition: service_healthy
+  healthcheck:
+    test:
+      - CMD
+      - wget
+      - --spider
+      - -q
+      - 0.0.0.0:8123/ping
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  ulimits:
+    nproc: 65535
+    nofile:
+      soft: 262144
+      hard: 262144
+  environment:
+    - CLICKHOUSE_SKIP_USER_SETUP=1
+x-zookeeper-defaults: &zookeeper-defaults
+  !!merge <<: *common
+  image: signoz/zookeeper:3.7.1
+  user: root
+  labels:
+    signoz.io/scrape: "true"
+    signoz.io/port: "9141"
+    signoz.io/path: "/metrics"
+  healthcheck:
+    test:
+      - CMD-SHELL
+      - curl -s -m 2 http://localhost:8080/commands/ruok | grep error | grep null
+    interval: 30s
+    timeout: 5s
+    retries: 3
+x-db-depend: &db-depend
+  !!merge <<: *common
+  depends_on:
+    clickhouse:
+      condition: service_healthy
+    clickhouse-2:
+      condition: service_healthy
+    clickhouse-3:
+      condition: service_healthy
+services:
+  init-clickhouse:
+    !!merge <<: *common
+    image: clickhouse/clickhouse-server:25.5.6
+    container_name: signoz-init-clickhouse
+    command:
+      - bash
+      - -c
+      - |
+        version="v0.0.1"
+        node_os=$$(uname -s | tr '[:upper:]' '[:lower:]')
+        node_arch=$$(uname -m | sed s/aarch64/arm64/ | sed s/x86_64/amd64/)
+        echo "Fetching histogram-binary for $${node_os}/$${node_arch}"
+        cd /tmp
+        wget -O histogram-quantile.tar.gz "https://github.com/SigNoz/signoz/releases/download/histogram-quantile%2F$${version}/histogram-quantile_$${node_os}_$${node_arch}.tar.gz"
+        tar -xvzf histogram-quantile.tar.gz
+        mv histogram-quantile /var/lib/clickhouse/user_scripts/histogramQuantile
+    restart: on-failure
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+  zookeeper-1:
+    !!merge <<: *zookeeper-defaults
+    container_name: signoz-zookeeper-1
+    # ports:
+    #   - "2181:2181"
+    #   - "2888:2888"
+    #   - "3888:3888"
+    volumes:
+      - zookeeper-1:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=1
+      - ZOO_SERVERS=0.0.0.0:2888:3888,zookeeper-2:2888:3888,zookeeper-3:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  zookeeper-2:
+    !!merge <<: *zookeeper-defaults
+    container_name: signoz-zookeeper-2
+    # ports:
+    #   - "2182:2181"
+    #   - "2889:2888"
+    #   - "3889:3888"
+    volumes:
+      - zookeeper-2:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=2
+      - ZOO_SERVERS=zookeeper-1:2888:3888,0.0.0.0:2888:3888,zookeeper-3:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  zookeeper-3:
+    !!merge <<: *zookeeper-defaults
+    container_name: signoz-zookeeper-3
+    # ports:
+    #   - "2183:2181"
+    #   - "2890:2888"
+    #   - "3890:3888"
+    volumes:
+      - zookeeper-3:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=3
+      - ZOO_SERVERS=zookeeper-1:2888:3888,zookeeper-2:2888:3888,0.0.0.0:2888:3888
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  clickhouse:
+    !!merge <<: *clickhouse-defaults
+    container_name: signoz-clickhouse
+    # ports:
+    #   - "9000:9000"
+    #   - "8123:8123"
+    #   - "9181:9181"
+    volumes:
+      - ../common/clickhouse/config.xml:/etc/clickhouse-server/config.xml
+      - ../common/clickhouse/users.xml:/etc/clickhouse-server/users.xml
+      - ../common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ../common/clickhouse/cluster.ha.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - clickhouse:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  clickhouse-2:
+    !!merge <<: *clickhouse-defaults
+    container_name: signoz-clickhouse-2
+    # ports:
+    #   - "9001:9000"
+    #   - "8124:8123"
+    #   - "9182:9181"
+    volumes:
+      - ../common/clickhouse/config.xml:/etc/clickhouse-server/config.xml
+      - ../common/clickhouse/users.xml:/etc/clickhouse-server/users.xml
+      - ../common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ../common/clickhouse/cluster.ha.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - clickhouse-2:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  clickhouse-3:
+    !!merge <<: *clickhouse-defaults
+    container_name: signoz-clickhouse-3
+    # ports:
+    #   - "9002:9000"
+    #   - "8125:8123"
+    #   - "9183:9181"
+    volumes:
+      - ../common/clickhouse/config.xml:/etc/clickhouse-server/config.xml
+      - ../common/clickhouse/users.xml:/etc/clickhouse-server/users.xml
+      - ../common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ../common/clickhouse/cluster.ha.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - clickhouse-3:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  signoz:
+    !!merge <<: *db-depend
+    image: signoz/signoz:${VERSION:-v0.129.0}
+    container_name: signoz
+    ports:
+      - "8080:8080" # signoz port
+    volumes:
+      - sqlite:/var/lib/signoz/
+    environment:
+      - SIGNOZ_ALERTMANAGER_PROVIDER=signoz
+      - SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
+      - SIGNOZ_TOKENIZER_JWT_SECRET=secret
+    healthcheck:
+      test:
+        - CMD
+        - wget
+        - --spider
+        - -q
+        - localhost:8080/api/v1/health
+      interval: 30s
+      timeout: 5s
+      retries: 3
+  otel-collector:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.5}
+    container_name: signoz-otel-collector
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate sync check &&
+        /signoz-otel-collector --config=/etc/otel-collector-config.yaml --manager-config=/etc/manager-config.yaml --copy-path=/var/tmp/collector-config.yaml
+    volumes:
+      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
+      - ../common/signoz/otel-collector-opamp-config.yaml:/etc/manager-config.yaml
+    environment:
+      - OTEL_RESOURCE_ATTRIBUTES=host.name=signoz-host,os.type=linux
+      - LOW_CARDINAL_EXCEPTION_GROUPING=false
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    ports:
+      # - "1777:1777"     # pprof extension
+      - "4317:4317" # OTLP gRPC receiver
+      - "4318:4318" # OTLP HTTP receiver
+  signoz-telemetrystore-migrator:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.5}
+    container_name: signoz-telemetrystore-migrator
+    environment:
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate bootstrap &&
+        /signoz-otel-collector migrate sync up &&
+        /signoz-otel-collector migrate async up
+    restart: on-failure
+networks:
+  signoz-net:
+    name: signoz-net
+volumes:
+  clickhouse:
+    name: signoz-clickhouse
+  clickhouse-2:
+    name: signoz-clickhouse-2
+  clickhouse-3:
+    name: signoz-clickhouse-3
+  sqlite:
+    name: signoz-sqlite
+  zookeeper-1:
+    name: signoz-zookeeper-1
+  zookeeper-2:
+    name: signoz-zookeeper-2
+  zookeeper-3:
+    name: signoz-zookeeper-3

deploy/docker/docker-compose.yaml

@@ -0,0 +1,185 @@
+version: "3"
+x-common: &common
+  networks:
+    - signoz-net
+  restart: unless-stopped
+  logging:
+    options:
+      max-size: 50m
+      max-file: "3"
+x-clickhouse-defaults: &clickhouse-defaults
+  !!merge <<: *common
+  image: clickhouse/clickhouse-server:25.5.6
+  tty: true
+  labels:
+    signoz.io/scrape: "true"
+    signoz.io/port: "9363"
+    signoz.io/path: "/metrics"
+  depends_on:
+    init-clickhouse:
+      condition: service_completed_successfully
+    zookeeper-1:
+      condition: service_healthy
+  healthcheck:
+    test:
+      - CMD
+      - wget
+      - --spider
+      - -q
+      - 0.0.0.0:8123/ping
+    interval: 30s
+    timeout: 5s
+    retries: 3
+  ulimits:
+    nproc: 65535
+    nofile:
+      soft: 262144
+      hard: 262144
+  environment:
+    - CLICKHOUSE_SKIP_USER_SETUP=1
+x-zookeeper-defaults: &zookeeper-defaults
+  !!merge <<: *common
+  image: signoz/zookeeper:3.7.1
+  user: root
+  labels:
+    signoz.io/scrape: "true"
+    signoz.io/port: "9141"
+    signoz.io/path: "/metrics"
+  healthcheck:
+    test:
+      - CMD-SHELL
+      - curl -s -m 2 http://localhost:8080/commands/ruok | grep error | grep null
+    interval: 30s
+    timeout: 5s
+    retries: 3
+x-db-depend: &db-depend
+  !!merge <<: *common
+  depends_on:
+    clickhouse:
+      condition: service_healthy
+services:
+  init-clickhouse:
+    !!merge <<: *common
+    image: clickhouse/clickhouse-server:25.5.6
+    container_name: signoz-init-clickhouse
+    command:
+      - bash
+      - -c
+      - |
+        version="v0.0.1"
+        node_os=$$(uname -s | tr '[:upper:]' '[:lower:]')
+        node_arch=$$(uname -m | sed s/aarch64/arm64/ | sed s/x86_64/amd64/)
+        echo "Fetching histogram-binary for $${node_os}/$${node_arch}"
+        cd /tmp
+        wget -O histogram-quantile.tar.gz "https://github.com/SigNoz/signoz/releases/download/histogram-quantile%2F$${version}/histogram-quantile_$${node_os}_$${node_arch}.tar.gz"
+        tar -xvzf histogram-quantile.tar.gz
+        mv histogram-quantile /var/lib/clickhouse/user_scripts/histogramQuantile
+    restart: on-failure
+    volumes:
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+  zookeeper-1:
+    !!merge <<: *zookeeper-defaults
+    container_name: signoz-zookeeper-1
+    # ports:
+    #   - "2181:2181"
+    #   - "2888:2888"
+    #   - "3888:3888"
+    volumes:
+      - zookeeper-1:/bitnami/zookeeper
+    environment:
+      - ZOO_SERVER_ID=1
+      - ALLOW_ANONYMOUS_LOGIN=yes
+      - ZOO_AUTOPURGE_INTERVAL=1
+      - ZOO_ENABLE_PROMETHEUS_METRICS=yes
+      - ZOO_PROMETHEUS_METRICS_PORT_NUMBER=9141
+  clickhouse:
+    !!merge <<: *clickhouse-defaults
+    container_name: signoz-clickhouse
+    # ports:
+    #   - "9000:9000"
+    #   - "8123:8123"
+    #   - "9181:9181"
+    volumes:
+      - ../common/clickhouse/config.xml:/etc/clickhouse-server/config.xml
+      - ../common/clickhouse/users.xml:/etc/clickhouse-server/users.xml
+      - ../common/clickhouse/custom-function.xml:/etc/clickhouse-server/custom-function.xml
+      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
+      - ../common/clickhouse/cluster.xml:/etc/clickhouse-server/config.d/cluster.xml
+      - clickhouse:/var/lib/clickhouse/
+      # - ../common/clickhouse/storage.xml:/etc/clickhouse-server/config.d/storage.xml
+  signoz:
+    !!merge <<: *db-depend
+    image: signoz/signoz:${VERSION:-v0.129.0}
+    container_name: signoz
+    ports:
+      - "8080:8080" # signoz port
+    volumes:
+      - sqlite:/var/lib/signoz/
+    environment:
+      - SIGNOZ_ALERTMANAGER_PROVIDER=signoz
+      - SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
+      - SIGNOZ_TOKENIZER_JWT_SECRET=secret
+    healthcheck:
+      test:
+        - CMD
+        - wget
+        - --spider
+        - -q
+        - localhost:8080/api/v1/health
+      interval: 30s
+      timeout: 5s
+      retries: 3
+  otel-collector:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.5}
+    container_name: signoz-otel-collector
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate sync check &&
+        /signoz-otel-collector --config=/etc/otel-collector-config.yaml --manager-config=/etc/manager-config.yaml --copy-path=/var/tmp/collector-config.yaml
+    volumes:
+      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
+      - ../common/signoz/otel-collector-opamp-config.yaml:/etc/manager-config.yaml
+    environment:
+      - OTEL_RESOURCE_ATTRIBUTES=host.name=signoz-host,os.type=linux
+      - LOW_CARDINAL_EXCEPTION_GROUPING=false
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    ports:
+      # - "1777:1777"     # pprof extension
+      - "4317:4317" # OTLP gRPC receiver
+      - "4318:4318" # OTLP HTTP receiver
+  signoz-telemetrystore-migrator:
+    !!merge <<: *db-depend
+    image: signoz/signoz-otel-collector:${OTELCOL_TAG:-v0.144.5}
+    container_name: signoz-telemetrystore-migrator
+    environment:
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_DSN=tcp://clickhouse:9000
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_CLUSTER=cluster
+      - SIGNOZ_OTEL_COLLECTOR_CLICKHOUSE_REPLICATION=true
+      - SIGNOZ_OTEL_COLLECTOR_TIMEOUT=10m
+    entrypoint:
+      - /bin/sh
+    command:
+      - -c
+      - |
+        /signoz-otel-collector migrate bootstrap &&
+        /signoz-otel-collector migrate sync up &&
+        /signoz-otel-collector migrate async up
+    restart: on-failure
+networks:
+  signoz-net:
+    name: signoz-net
+volumes:
+  clickhouse:
+    name: signoz-clickhouse
+  sqlite:
+    name: signoz-sqlite
+  zookeeper-1:
+    name: signoz-zookeeper-1

deploy/docker/otel-collector-config.yaml

@@ -0,0 +1,118 @@
+connectors:
+  signozmeter:
+    metrics_flush_interval: 1h
+    dimensions:
+      - name: service.name
+      - name: deployment.environment
+      - name: host.name
+receivers:
+  otlp:
+    protocols:
+      grpc:
+        endpoint: 0.0.0.0:4317
+      http:
+        endpoint: 0.0.0.0:4318
+  prometheus:
+    config:
+      global:
+        scrape_interval: 60s
+      scrape_configs:
+        - job_name: otel-collector
+          static_configs:
+          - targets:
+              - localhost:8888
+            labels:
+              job_name: otel-collector
+processors:
+  batch:
+    send_batch_size: 10000
+    send_batch_max_size: 11000
+    timeout: 10s
+  batch/meter:
+    send_batch_max_size: 25000
+    send_batch_size: 20000
+    timeout: 1s
+  resourcedetection:
+    # Using OTEL_RESOURCE_ATTRIBUTES envvar, env detector adds custom labels.
+    detectors: [env, system]
+    timeout: 2s
+  signozspanmetrics/delta:
+    metrics_exporter: signozclickhousemetrics
+    metrics_flush_interval: 60s
+    latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ]
+    dimensions_cache_size: 100000
+    aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
+    enable_exp_histogram: true
+    dimensions:
+      - name: service.namespace
+        default: default
+      - name: deployment.environment
+        default: default
+      # This is added to ensure the uniqueness of the timeseries
+      # Otherwise, identical timeseries produced by multiple replicas of
+      # collectors result in incorrect APM metrics
+      - name: signoz.collector.id
+      - name: service.version
+      - name: browser.platform
+      - name: browser.mobile
+      - name: k8s.cluster.name
+      - name: k8s.node.name
+      - name: k8s.namespace.name
+      - name: host.name
+      - name: host.type
+      - name: container.name
+extensions:
+  health_check:
+    endpoint: 0.0.0.0:13133
+  pprof:
+    endpoint: 0.0.0.0:1777
+exporters:
+  clickhousetraces:
+    datasource: tcp://clickhouse:9000/signoz_traces
+    low_cardinal_exception_grouping: ${env:LOW_CARDINAL_EXCEPTION_GROUPING}
+    use_new_schema: true
+  signozclickhousemetrics:
+    dsn: tcp://clickhouse:9000/signoz_metrics
+  clickhouselogsexporter:
+    dsn: tcp://clickhouse:9000/signoz_logs
+    timeout: 10s
+    use_new_schema: true
+  signozclickhousemeter:
+    dsn: tcp://clickhouse:9000/signoz_meter
+    timeout: 45s
+    sending_queue:
+      enabled: false
+  metadataexporter:
+    cache:
+      provider: in_memory
+    dsn: tcp://clickhouse:9000/signoz_metadata
+    enabled: true
+    timeout: 45s
+service:
+  telemetry:
+    logs:
+      encoding: json
+  extensions:
+    - health_check
+    - pprof
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: [signozspanmetrics/delta, batch]
+      exporters: [clickhousetraces, metadataexporter, signozmeter]
+    metrics:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [signozclickhousemetrics, metadataexporter, signozmeter]
+    metrics/prometheus:
+      receivers: [prometheus]
+      processors: [batch]
+      exporters: [signozclickhousemetrics, metadataexporter, signozmeter]
+    logs:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [clickhouselogsexporter, metadataexporter, signozmeter]
+    metrics/meter:
+      receivers: [signozmeter]
+      processors: [batch/meter]
+      exporters: [signozclickhousemeter]

deploy/install.sh

@@ -0,0 +1,563 @@
+#!/bin/bash
+
+set -o errexit
+
+# Variables
+BASE_DIR="$(dirname "$(readlink -f "$0")")"
+DOCKER_STANDALONE_DIR="docker"
+DOCKER_SWARM_DIR="docker-swarm" # TODO: Add docker swarm support
+
+# Regular Colors
+Black='\033[0;30m'        # Black
+Red='\[\e[0;31m\]'        # Red
+Green='\033[0;32m'        # Green
+Yellow='\033[0;33m'       # Yellow
+Blue='\033[0;34m'         # Blue
+Purple='\033[0;35m'       # Purple
+Cyan='\033[0;36m'         # Cyan
+White='\033[0;37m'        # White
+NC='\033[0m' # No Color
+
+is_command_present() {
+    type "$1" >/dev/null 2>&1
+}
+
+# Check whether 'wget' command exists.
+has_wget() {
+    has_cmd wget
+}
+
+# Check whether 'curl' command exists.
+has_curl() {
+    has_cmd curl
+}
+
+# Check whether the given command exists.
+has_cmd() {
+    command -v "$1" > /dev/null 2>&1
+}
+
+# Check if docker compose plugin is present
+has_docker_compose_plugin() {
+    docker compose version > /dev/null 2>&1
+}
+
+is_mac() {
+    [[ $OSTYPE == darwin* ]]
+}
+
+is_arm64(){
+    [[ `uname -m` == 'arm64' || `uname -m` == 'aarch64' ]]
+}
+
+check_os() {
+    if is_mac; then
+        package_manager="brew"
+        desired_os=1
+        os="Mac"
+        return
+    fi
+
+    if is_arm64; then
+        arch="arm64"
+        arch_official="aarch64"
+    else
+        arch="amd64"
+        arch_official="x86_64"
+    fi
+
+    platform=$(uname -s | tr '[:upper:]' '[:lower:]')
+
+    os_name="$(cat /etc/*-release | awk -F= '$1 == "NAME" { gsub(/"/, ""); print $2; exit }')"
+
+    case "$os_name" in
+        Ubuntu*|Pop!_OS)
+            desired_os=1
+            os="ubuntu"
+            package_manager="apt-get"
+            ;;
+        Amazon\ Linux*)
+            desired_os=1
+            os="amazon linux"
+            package_manager="yum"
+            ;;
+        Debian*)
+            desired_os=1
+            os="debian"
+            package_manager="apt-get"
+            ;;
+        Linux\ Mint*)
+            desired_os=1
+            os="linux mint"
+            package_manager="apt-get"
+            ;;
+        Red\ Hat*)
+            desired_os=1
+            os="rhel"
+            package_manager="yum"
+            ;;
+        CentOS*)
+            desired_os=1
+            os="centos"
+            package_manager="yum"
+            ;;
+        Rocky*)
+            desired_os=1
+            os="centos"
+            package_manager="yum"
+            ;;
+        SLES*)
+            desired_os=1
+            os="sles"
+            package_manager="zypper"
+            ;;
+        openSUSE*)
+            desired_os=1
+            os="opensuse"
+            package_manager="zypper"
+            ;;
+        *)
+            desired_os=0
+            os="Not Found: $os_name"
+    esac
+}
+
+
+# This function checks if the relevant ports required by SigNoz are available or not
+# The script should error out in case they aren't available
+check_ports_occupied() {
+    local port_check_output
+    local ports_pattern="8080|4317"
+
+    if is_mac; then
+        port_check_output="$(netstat -anp tcp | awk '$6 == "LISTEN" && $4 ~ /^.*\.('"$ports_pattern"')$/')"
+    elif is_command_present ss; then
+        # The `ss` command seems to be a better/faster version of `netstat`, but is not available on all Linux
+        # distributions by default. Other distributions have `ss` but no `netstat`. So, we try for `ss` first, then
+        # fallback to `netstat`.
+        port_check_output="$(ss --all --numeric --tcp | awk '$1 == "LISTEN" && $4 ~ /^.*:('"$ports_pattern"')$/')"
+    elif is_command_present netstat; then
+        port_check_output="$(netstat --all --numeric --tcp | awk '$6 == "LISTEN" && $4 ~ /^.*:('"$ports_pattern"')$/')"
+    fi
+
+    if [[ -n $port_check_output ]]; then
+        send_event "port_not_available"
+
+        echo "+++++++++++ ERROR ++++++++++++++++++++++"
+        echo "SigNoz requires ports 8080 & 4317 to be open. Please shut down any other service(s) that may be running on these ports."
+        echo "You can run SigNoz on another port following this guide https://signoz.io/docs/install/troubleshooting/"
+        echo "++++++++++++++++++++++++++++++++++++++++"
+        echo ""
+        exit 1
+    fi
+}
+
+install_docker() {
+    echo "++++++++++++++++++++++++"
+    echo "Setting up docker repos"
+
+
+    if [[ $package_manager == apt-get ]]; then
+        apt_cmd="$sudo_cmd apt-get --yes --quiet"
+        $apt_cmd update
+        $apt_cmd install software-properties-common gnupg-agent
+        curl -fsSL "https://download.docker.com/linux/$os/gpg" | $sudo_cmd apt-key add -
+        $sudo_cmd add-apt-repository \
+            "deb [arch=$arch] https://download.docker.com/linux/$os $(lsb_release -cs) stable"
+        $apt_cmd update
+        echo "Installing docker"
+        $apt_cmd install docker-ce docker-ce-cli containerd.io
+    elif [[ $package_manager == zypper ]]; then
+        zypper_cmd="$sudo_cmd zypper --quiet --no-gpg-checks --non-interactive"
+        echo "Installing docker"
+        if [[ $os == sles ]]; then
+            os_sp="$(cat /etc/*-release | awk -F= '$1 == "VERSION_ID" { gsub(/"/, ""); print $2; exit }')"
+            os_arch="$(uname -i)"
+            SUSEConnect -p sle-module-containers/$os_sp/$os_arch -r ''
+        fi
+        $zypper_cmd install docker docker-runc containerd
+        $sudo_cmd systemctl enable docker.service
+    elif [[ $package_manager == yum && $os == 'amazon linux' ]]; then
+        echo
+        echo "Amazon Linux detected ... "
+        echo
+        # yum install docker
+        # service docker start
+        $sudo_cmd yum install -y amazon-linux-extras
+        $sudo_cmd amazon-linux-extras enable docker
+        $sudo_cmd yum install -y docker
+    else
+
+        yum_cmd="$sudo_cmd yum --assumeyes --quiet"
+        $yum_cmd install yum-utils
+        $sudo_cmd yum-config-manager --add-repo https://download.docker.com/linux/$os/docker-ce.repo
+        echo "Installing docker"
+        $yum_cmd install docker-ce docker-ce-cli containerd.io
+    fi
+}
+
+compose_version () {
+    local compose_version
+    compose_version="$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep 'tag_name' | cut -d\" -f4)"
+    echo "${compose_version:-v2.18.1}"
+}
+
+install_docker_compose() {
+    if [[ $package_manager == "apt-get" || $package_manager == "zypper" || $package_manager == "yum" ]]; then
+        if [[ ! -f /usr/bin/docker-compose ]];then
+            echo "++++++++++++++++++++++++"
+            echo "Installing docker-compose"
+            compose_url="https://github.com/docker/compose/releases/download/$(compose_version)/docker-compose-$platform-$arch_official"
+            echo "Downloading docker-compose from $compose_url"
+            $sudo_cmd curl -L "$compose_url" -o /usr/local/bin/docker-compose
+            $sudo_cmd chmod +x /usr/local/bin/docker-compose
+            $sudo_cmd ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
+            echo "docker-compose installed!"
+            echo ""
+        fi
+    else
+        send_event "docker_compose_not_found"
+
+        echo "+++++++++++ IMPORTANT READ ++++++++++++++++++++++"
+        echo "docker-compose not found! Please install docker-compose first and then continue with this installation."
+        echo "Refer https://docs.docker.com/compose/install/ for installing docker-compose."
+        echo "+++++++++++++++++++++++++++++++++++++++++++++++++"
+        exit 1
+    fi
+}
+
+start_docker() {
+    echo -e "🐳 Starting Docker ...\n"
+    if [[ $os == "Mac" ]]; then
+        open --background -a Docker && while ! docker system info > /dev/null 2>&1; do sleep 1; done
+    else
+        if ! $sudo_cmd systemctl is-active docker.service > /dev/null; then
+            echo "Starting docker service"
+            $sudo_cmd systemctl start docker.service
+        fi
+        if [[ -z $sudo_cmd ]]; then
+            if ! docker ps > /dev/null && true; then
+                request_sudo
+            fi
+        fi
+    fi
+}
+
+wait_for_containers_start() {
+    local timeout=$1
+
+    # The while loop is important because for-loops don't work for dynamic values
+    while [[ $timeout -gt 0 ]]; do
+        status_code="$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:8080/api/v1/health?live=1" || true)"
+        if [[ status_code -eq 200 ]]; then
+            break
+        else
+            echo -ne "Waiting for all containers to start. This check will timeout in $timeout seconds ...\r\c"
+        fi
+        ((timeout--))
+        sleep 1
+    done
+
+    echo ""
+}
+
+bye() {  # Prints a friendly good bye message and exits the script.
+    # Switch back to the original directory
+    popd > /dev/null 2>&1
+    if [[ "$?" -ne 0 ]]; then
+        set +o errexit
+
+        echo "🔴 The containers didn't seem to start correctly. Please run the following command to check containers that may have errored out:"
+        echo ""
+        echo -e "cd ${DOCKER_STANDALONE_DIR}"
+        echo -e "$sudo_cmd $docker_compose_cmd ps -a"
+
+        echo "Please read our troubleshooting guide https://signoz.io/docs/install/troubleshooting/"
+        echo "or reach us for support in #help channel in our Slack Community https://signoz.io/slack"
+        echo "++++++++++++++++++++++++++++++++++++++++"
+
+        if [[ $email == "" ]]; then
+            echo -e "\n📨 Please share your email to receive support with the installation"
+            read -rp 'Email: ' email
+
+            while [[ $email == "" ]]
+            do
+                read -rp 'Email: ' email
+            done
+        fi
+
+        send_event "installation_support"
+
+
+        echo ""
+        echo -e "\nWe will reach out to you at the email provided shortly, Exiting for now. Bye! 👋 \n"
+        exit 0
+    fi
+}
+
+request_sudo() {
+    if hash sudo 2>/dev/null; then
+        echo -e "\n\n🙇 We will need sudo access to complete the installation."
+        if (( $EUID != 0 )); then
+            sudo_cmd="sudo"
+            echo -e "Please enter your sudo password, if prompted."
+            if ! $sudo_cmd -l | grep -e "NOPASSWD: ALL" > /dev/null && ! $sudo_cmd -v; then
+                echo "Need sudo privileges to proceed with the installation."
+                exit 1;
+            fi
+
+            echo -e "Got it! Thanks!! 🙏\n"
+            echo -e "Okay! We will bring up the SigNoz cluster from here 🚀\n"
+        fi
+	fi
+}
+
+echo ""
+echo -e "👋 Thank you for trying out SigNoz! "
+echo ""
+
+sudo_cmd=""
+docker_compose_cmd=""
+
+# Check sudo permissions
+if (( $EUID != 0 )); then
+    echo "🟡 Running installer with non-sudo permissions."
+    echo "   In case of any failure or prompt, please consider running the script with sudo privileges."
+    echo ""
+else
+    sudo_cmd="sudo"
+fi
+
+# Checking OS and assigning package manager
+desired_os=0
+os=""
+email=""
+echo -e "🌏 Detecting your OS ...\n"
+check_os
+
+# Obtain unique installation id
+# sysinfo="$(uname -a)"
+# if [[ $? -ne 0 ]]; then
+#     uuid="$(uuidgen)"
+#     uuid="${uuid:-$(cat /proc/sys/kernel/random/uuid)}"
+#     sysinfo="${uuid:-$(cat /proc/sys/kernel/random/uuid)}"
+# fi
+if ! sysinfo="$(uname -a)"; then
+    uuid="$(uuidgen)"
+    uuid="${uuid:-$(cat /proc/sys/kernel/random/uuid)}"
+    sysinfo="${uuid:-$(cat /proc/sys/kernel/random/uuid)}"
+fi
+
+digest_cmd=""
+if hash shasum 2>/dev/null; then
+    digest_cmd="shasum -a 256"
+elif hash sha256sum 2>/dev/null; then
+    digest_cmd="sha256sum"
+elif hash openssl 2>/dev/null; then
+    digest_cmd="openssl dgst -sha256"
+fi
+
+if [[ -z $digest_cmd ]]; then
+    SIGNOZ_INSTALLATION_ID="$sysinfo"
+else
+    SIGNOZ_INSTALLATION_ID=$(echo "$sysinfo" | $digest_cmd | grep -E -o '[a-zA-Z0-9]{64}')
+fi
+
+setup_type='clickhouse'
+
+# Run bye if failure happens
+trap bye EXIT
+
+URL="https://api.segment.io/v1/track"
+HEADER_1="Content-Type: application/json"
+HEADER_2="Authorization: Basic OWtScko3b1BDR1BFSkxGNlFqTVBMdDVibGpGaFJRQnI="
+
+send_event() {
+    error=""
+
+    case "$1" in
+        'install_started')
+            event="Installation Started"
+            ;;
+        'os_not_supported')
+            event="Installation Error"
+            error="OS Not Supported"
+            ;;
+        'docker_not_installed')
+            event="Installation Error"
+            error="Docker not installed"
+            ;;
+        'docker_compose_not_found')
+            event="Installation Error"
+            event="Docker Compose not found"
+            ;;
+        'port_not_available')
+            event="Installation Error"
+            error="port not available"
+            ;;
+        'installation_error_checks')
+            event="Installation Error - Checks"
+            error="Containers not started"
+            others='"data": "some_checks",'
+            ;;
+        'installation_support')
+            event="Installation Support"
+            others='"email": "'"$email"'",'
+            ;;
+        'installation_success')
+            event="Installation Success"
+            ;;
+        'identify_successful_installation')
+            event="Identify Successful Installation"
+            others='"email": "'"$email"'",'
+            ;;
+        *)
+            print_error "unknown event type: $1"
+            exit 1
+            ;;
+    esac
+
+    if [[ "$error" != "" ]]; then
+        error='"error": "'"$error"'", '
+    fi
+
+    DATA='{ "anonymousId": "'"$SIGNOZ_INSTALLATION_ID"'", "event": "'"$event"'", "properties": { "os": "'"$os"'", '"$error $others"' "setup_type": "'"$setup_type"'" } }'
+
+    if has_curl; then
+        curl -sfL -d "$DATA" --header "$HEADER_1" --header "$HEADER_2" "$URL" > /dev/null 2>&1
+    elif has_wget; then
+        wget -q --post-data="$DATA" --header "$HEADER_1" --header "$HEADER_2" "$URL" > /dev/null 2>&1
+    fi
+}
+
+send_event "install_started"
+
+if [[ $desired_os -eq 0 ]]; then
+    send_event "os_not_supported"
+fi
+
+# Check is Docker daemon is installed and available. If not, the install & start Docker for Linux machines. We cannot automatically install Docker Desktop on Mac OS
+if ! is_command_present docker; then
+
+    if [[ $package_manager == "apt-get" || $package_manager == "zypper" || $package_manager == "yum" ]]; then
+        request_sudo
+        install_docker
+        # enable docker without sudo from next reboot
+        sudo usermod -aG docker "${USER}"
+    elif is_mac; then
+        echo ""
+        echo "+++++++++++ IMPORTANT READ ++++++++++++++++++++++"
+        echo "Docker Desktop must be installed manually on Mac OS to proceed. Docker can only be installed automatically on Ubuntu / openSUSE / SLES / Redhat / Cent OS"
+        echo "https://docs.docker.com/docker-for-mac/install/"
+        echo "++++++++++++++++++++++++++++++++++++++++++++++++"
+
+        send_event "docker_not_installed"
+        exit 1
+    else
+        echo ""
+        echo "+++++++++++ IMPORTANT READ ++++++++++++++++++++++"
+        echo "Docker must be installed manually on your machine to proceed. Docker can only be installed automatically on Ubuntu / openSUSE / SLES / Redhat / Cent OS"
+        echo "https://docs.docker.com/get-docker/"
+        echo "++++++++++++++++++++++++++++++++++++++++++++++++"
+
+        send_event "docker_not_installed"
+        exit 1
+    fi
+fi
+
+if has_docker_compose_plugin; then
+    echo "docker compose plugin is present, using it"
+    docker_compose_cmd="docker compose"
+# Install docker-compose
+else
+    docker_compose_cmd="docker-compose"
+    if ! is_command_present docker-compose; then
+        request_sudo
+        install_docker_compose
+    fi
+fi
+
+start_docker
+
+# Switch to the Docker Standalone directory
+pushd "${BASE_DIR}/${DOCKER_STANDALONE_DIR}" > /dev/null 2>&1
+
+# check for open ports, if signoz is not installed
+if is_command_present docker-compose; then
+    if $sudo_cmd $docker_compose_cmd ps | grep "signoz" | grep -q "healthy" > /dev/null 2>&1; then
+        echo "SigNoz already installed, skipping the occupied ports check"
+    else
+        check_ports_occupied
+    fi
+fi
+
+echo ""
+echo -e "\n🟡 Pulling the latest container images for SigNoz.\n"
+$sudo_cmd $docker_compose_cmd pull
+
+echo ""
+echo "🟡 Starting the SigNoz containers. It may take a few minutes ..."
+echo
+# The $docker_compose_cmd command does some nasty stuff for the `--detach` functionality. So we add a `|| true` so that the
+# script doesn't exit because this command looks like it failed to do it's thing.
+$sudo_cmd $docker_compose_cmd up --detach --remove-orphans || true
+
+wait_for_containers_start 60
+echo ""
+
+if [[ $status_code -ne 200 ]]; then
+    echo "+++++++++++ ERROR ++++++++++++++++++++++"
+    echo "🔴 The containers didn't seem to start correctly. Please run the following command to check containers that may have errored out:"
+    echo ""
+
+    echo "cd ${DOCKER_STANDALONE_DIR}"
+    echo "$sudo_cmd $docker_compose_cmd ps -a"
+    echo ""
+
+    echo "Try bringing down the containers and retrying the installation"
+    echo "cd ${DOCKER_STANDALONE_DIR}"
+    echo "$sudo_cmd $docker_compose_cmd down -v"
+    echo ""
+
+    echo "Please read our troubleshooting guide https://signoz.io/docs/install/troubleshooting/"
+    echo "or reach us on SigNoz for support https://signoz.io/slack"
+    echo "++++++++++++++++++++++++++++++++++++++++"
+
+    send_event "installation_error_checks"
+    exit 1
+
+else
+    send_event "installation_success"
+
+    echo "++++++++++++++++++ SUCCESS ++++++++++++++++++++++"
+    echo ""
+    echo "🟢 Your installation is complete!"
+    echo ""
+    echo -e "🟢 SigNoz is running on http://localhost:8080"
+    echo ""
+    echo "ℹ️  By default, retention period is set to 15 days for logs and traces, and 30 days for metrics."
+    echo -e "To change this, navigate to the General tab on the Settings page of SigNoz UI. For more details, refer to https://signoz.io/docs/userguide/retention-period \n"
+
+    echo "ℹ️  To bring down SigNoz and clean volumes:"
+    echo ""
+    echo "cd ${DOCKER_STANDALONE_DIR}"
+    echo "$sudo_cmd $docker_compose_cmd down -v"
+
+    echo ""
+    echo "+++++++++++++++++++++++++++++++++++++++++++++++++"
+    echo ""
+    echo "👉 Need help in Getting Started?"
+    echo -e "Join us on Slack https://signoz.io/slack"
+    echo ""
+    echo -e "\n📨 Please share your email to receive support & updates about SigNoz!"
+    read -rp 'Email: ' email
+
+    while [[ $email == "" ]]
+    do
+        read -rp 'Email: ' email
+    done
+
+    send_event "identify_successful_installation"
+fi
+
+echo -e "\n🙏 Thank you!\n"

docs/api/openapi.yml

@@ -2437,17 +2437,6 @@ components:
         url:
           type: string
       type: object
-    DashboardTextVariableSpec:
-      properties:
-        constant:
-          type: boolean
-        display:
-          $ref: '#/components/schemas/VariableDisplay'
-        name:
-          type: string
-        value:
-          type: string
-      type: object
     DashboardtypesAxes:
       properties:
         isLogScale:
@@ -2812,9 +2801,15 @@ components:
             type: string
           nullable: true
           type: object
+        mode:
+          $ref: '#/components/schemas/DashboardtypesLegendMode'
         position:
           $ref: '#/components/schemas/DashboardtypesLegendPosition'
       type: object
+    DashboardtypesLegendMode:
+      enum:
+      - list
+      type: string
     DashboardtypesLegendPosition:
       enum:
       - bottom
@@ -2865,15 +2860,25 @@ components:
         display:
           $ref: '#/components/schemas/DashboardtypesDisplay'
         name:
+          minLength: 1
           type: string
         plugin:
           $ref: '#/components/schemas/DashboardtypesVariablePlugin'
         sort:
-          nullable: true
-          type: string
+          $ref: '#/components/schemas/DashboardtypesListVariableSpecSort'
       required:
-      - display
+      - name
       type: object
+    DashboardtypesListVariableSpecSort:
+      enum:
+      - none
+      - alphabetical-asc
+      - alphabetical-desc
+      - numerical-asc
+      - numerical-desc
+      - alphabetical-ci-asc
+      - alphabetical-ci-desc
+      type: string
     DashboardtypesListableDashboardForUserV2:
       properties:
         dashboards:
@@ -3383,8 +3388,13 @@ components:
     DashboardtypesSpanGaps:
       properties:
         fillLessThan:
+          description: The maximum gap size to connect when fillOnlyBelow is true.
+            Gaps larger than this duration are left disconnected.
           type: string
         fillOnlyBelow:
+          description: Controls whether lines connect across null values. When false
+            (default), all gaps are connected. When true, only gaps smaller than fillLessThan
+            are connected.
           type: boolean
       type: object
     DashboardtypesStorableDashboardData:
@@ -3432,6 +3442,20 @@ components:
       - color
       - columnName
       type: object
+    DashboardtypesTextVariableSpec:
+      properties:
+        constant:
+          type: boolean
+        display:
+          $ref: '#/components/schemas/DashboardtypesDisplay'
+        name:
+          minLength: 1
+          type: string
+        value:
+          type: string
+      required:
+      - name
+      type: object
     DashboardtypesThresholdFormat:
       enum:
       - text
@@ -3451,7 +3475,6 @@ components:
       required:
       - value
       - color
-      - label
       type: object
     DashboardtypesTimePreference:
       enum:
@@ -3536,23 +3559,11 @@ components:
       discriminator:
         mapping:
           ListVariable: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpec'
-          TextVariable: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpec'
+          TextVariable: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpec'
         propertyName: kind
       oneOf:
       - $ref: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpec'
-      - $ref: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpec'
-      type: object
-    DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpec:
-      properties:
-        kind:
-          enum:
-          - TextVariable
-          type: string
-        spec:
-          $ref: '#/components/schemas/DashboardTextVariableSpec'
-      required:
-      - kind
-      - spec
+      - $ref: '#/components/schemas/DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpec'
       type: object
     DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpec:
       properties:
@@ -3566,6 +3577,18 @@ components:
       - kind
       - spec
       type: object
+    DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpec:
+      properties:
+        kind:
+          enum:
+          - TextVariable
+          type: string
+        spec:
+          $ref: '#/components/schemas/DashboardtypesTextVariableSpec'
+      required:
+      - kind
+      - spec
+      type: object
     DashboardtypesVariablePlugin:
       discriminator:
         mapping:
@@ -7651,15 +7674,6 @@ components:
       type: object
     VariableDefaultValue:
       type: object
-    VariableDisplay:
-      properties:
-        description:
-          type: string
-        hidden:
-          type: boolean
-        name:
-          type: string
-      type: object
     ZeustypesGettableHost:
       properties:
         hosts:

frontend/src/api/generated/services/sigNoz.schemas.ts

@@ -3154,37 +3154,6 @@ export interface DashboardLinkDTO {
 	url?: string;
 }
 
-export interface VariableDisplayDTO {
-	/**
-	 * @type string
-	 */
-	description?: string;
-	/**
-	 * @type boolean
-	 */
-	hidden?: boolean;
-	/**
-	 * @type string
-	 */
-	name?: string;
-}
-
-export interface DashboardTextVariableSpecDTO {
-	/**
-	 * @type boolean
-	 */
-	constant?: boolean;
-	display?: VariableDisplayDTO;
-	/**
-	 * @type string
-	 */
-	name?: string;
-	/**
-	 * @type string
-	 */
-	value?: string;
-}
-
 export interface DashboardtypesAxesDTO {
 	/**
 	 * @type boolean
@@ -3216,6 +3185,9 @@ export interface DashboardtypesPanelFormattingDTO {
 	unit?: string;
 }
 
+export enum DashboardtypesLegendModeDTO {
+	list = 'list',
+}
 export enum DashboardtypesLegendPositionDTO {
 	bottom = 'bottom',
 	right = 'right',
@@ -3235,6 +3207,7 @@ export interface DashboardtypesLegendDTO {
 	 * @type object,null
 	 */
 	customColors?: DashboardtypesLegendDTOCustomColors;
+	mode?: DashboardtypesLegendModeDTO;
 	position?: DashboardtypesLegendPositionDTO;
 }
 
@@ -3246,7 +3219,7 @@ export interface DashboardtypesThresholdWithLabelDTO {
 	/**
 	 * @type string
 	 */
-	label: string;
+	label?: string;
 	/**
 	 * @type string
 	 */
@@ -3911,10 +3884,12 @@ export enum DashboardtypesLineStyleDTO {
 export interface DashboardtypesSpanGapsDTO {
 	/**
 	 * @type string
+	 * @description The maximum gap size to connect when fillOnlyBelow is true. Gaps larger than this duration are left disconnected.
 	 */
 	fillLessThan?: string;
 	/**
 	 * @type boolean
+	 * @description Controls whether lines connect across null values. When false (default), all gaps are connected. When true, only gaps smaller than fillLessThan are connected.
 	 */
 	fillOnlyBelow?: boolean;
 }
@@ -4546,6 +4521,15 @@ export type DashboardtypesVariablePluginDTO =
 	| DashboardtypesVariablePluginVariantGithubComSigNozSignozPkgTypesDashboardtypesQueryVariableSpecDTO
 	| DashboardtypesVariablePluginVariantGithubComSigNozSignozPkgTypesDashboardtypesCustomVariableSpecDTO;
 
+export enum DashboardtypesListVariableSpecSortDTO {
+	none = 'none',
+	'alphabetical-asc' = 'alphabetical-asc',
+	'alphabetical-desc' = 'alphabetical-desc',
+	'numerical-asc' = 'numerical-asc',
+	'numerical-desc' = 'numerical-desc',
+	'alphabetical-ci-asc' = 'alphabetical-ci-asc',
+	'alphabetical-ci-desc' = 'alphabetical-ci-desc',
+}
 export interface DashboardtypesListVariableSpecDTO {
 	/**
 	 * @type boolean
@@ -4564,16 +4548,14 @@ export interface DashboardtypesListVariableSpecDTO {
 	 */
 	customAllValue?: string;
 	defaultValue?: VariableDefaultValueDTO;
-	display: DashboardtypesDisplayDTO;
+	display?: DashboardtypesDisplayDTO;
 	/**
 	 * @type string
+	 * @minLength 1
 	 */
-	name?: string;
+	name: string;
 	plugin?: DashboardtypesVariablePluginDTO;
-	/**
-	 * @type string,null
-	 */
-	sort?: string | null;
+	sort?: DashboardtypesListVariableSpecSortDTO;
 }
 
 export interface DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpecDTO {
@@ -4585,21 +4567,38 @@ export interface DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDash
 	spec: DashboardtypesListVariableSpecDTO;
 }
 
-export enum DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpecDTOKind {
+export enum DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpecDTOKind {
 	TextVariable = 'TextVariable',
 }
-export interface DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpecDTO {
+export interface DashboardtypesTextVariableSpecDTO {
+	/**
+	 * @type boolean
+	 */
+	constant?: boolean;
+	display?: DashboardtypesDisplayDTO;
+	/**
+	 * @type string
+	 * @minLength 1
+	 */
+	name: string;
+	/**
+	 * @type string
+	 */
+	value?: string;
+}
+
+export interface DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpecDTO {
 	/**
 	 * @enum TextVariable
 	 * @type string
 	 */
-	kind: DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpecDTOKind;
-	spec: DashboardTextVariableSpecDTO;
+	kind: DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpecDTOKind;
+	spec: DashboardtypesTextVariableSpecDTO;
 }
 
 export type DashboardtypesVariableDTO =
 	| DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpecDTO
-	| DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpecDTO;
+	| DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpecDTO;
 
 export interface DashboardtypesDashboardSpecDTO {
 	/**

frontend/src/pages/DashboardPageV2/DashboardContainer/DashboardSettings/Variables/VariableForm/QueryVariableFields.tsx

@@ -2,15 +2,16 @@ import { useState } from 'react';
 import { Button } from '@signozhq/ui/button';
 import { Typography } from '@signozhq/ui/typography';
 import dashboardVariablesQuery from 'api/dashboard/variables/dashboardVariablesQuery';
+import { DashboardtypesListVariableSpecSortDTO as VariableSortDTO } from 'api/generated/services/sigNoz.schemas';
 import Editor from 'components/Editor';
 import sortValues from 'lib/dashboardVariables/sortVariableValues';
 
-import type { VariableSort } from '../variableModel';
+import { sortDirectionOf } from '../variableModel';
 import styles from './VariableForm.module.scss';
 
 interface QueryVariableFieldsProps {
 	queryValue: string;
-	sort: VariableSort;
+	sort: VariableSortDTO;
 	onChange: (queryValue: string) => void;
 	onPreview: (values: (string | number)[]) => void;
 	onError: (message: string | null) => void;
@@ -36,7 +37,10 @@ function QueryVariableFields({
 			});
 			if (res.statusCode === 200 && res.payload) {
 				onPreview(
-					sortValues(res.payload.variableValues ?? [], sort) as (string | number)[],
+					sortValues(res.payload.variableValues ?? [], sortDirectionOf(sort)) as (
+						| string
+						| number
+					)[],
 				);
 			} else {
 				onError(res.error || 'Failed to run query');

frontend/src/pages/DashboardPageV2/DashboardContainer/DashboardSettings/Variables/VariableForm/VariableForm.tsx

@@ -12,10 +12,12 @@ import { Collapse, Input as AntdInput, Select } from 'antd';
 import { commaValuesParser } from 'lib/dashboardVariables/customCommaValuesParser';
 import sortValues from 'lib/dashboardVariables/sortVariableValues';
 
+import { DashboardtypesListVariableSpecSortDTO as VariableSortDTO } from 'api/generated/services/sigNoz.schemas';
+
 import {
+	sortDirectionOf,
 	VARIABLE_SORTS,
 	type VariableFormModel,
-	type VariableSort,
 	type VariableType,
 } from '../variableModel';
 import DynamicVariableFields from './DynamicVariableFields';
@@ -23,10 +25,16 @@ import QueryVariableFields from './QueryVariableFields';
 import VariableTypeSelector from './VariableTypeSelector';
 import styles from './VariableForm.module.scss';
 
-const SORT_LABEL: Record<VariableSort, string> = {
-	DISABLED: 'Disabled',
-	ASC: 'Ascending',
-	DESC: 'Descending',
+const SORT_LABEL: Record<VariableSortDTO, string> = {
+	[VariableSortDTO.none]: 'Disabled',
+	[VariableSortDTO['alphabetical-asc']]: 'Alphabetical (asc)',
+	[VariableSortDTO['alphabetical-desc']]: 'Alphabetical (desc)',
+	[VariableSortDTO['numerical-asc']]: 'Numerical (asc)',
+	[VariableSortDTO['numerical-desc']]: 'Numerical (desc)',
+	[VariableSortDTO['alphabetical-ci-asc']]:
+		'Alphabetical, case-insensitive (asc)',
+	[VariableSortDTO['alphabetical-ci-desc']]:
+		'Alphabetical, case-insensitive (desc)',
 };
 
 function getNameError(name: string, existingNames: string[]): string | null {
@@ -91,7 +99,10 @@ function VariableForm({
 	const onCustomChange = (value: string): void => {
 		set({ customValue: value });
 		setPreviewValues(
-			sortValues(commaValuesParser(value), model.sort) as (string | number)[],
+			sortValues(commaValuesParser(value), sortDirectionOf(model.sort)) as (
+				| string
+				| number
+			)[],
 		);
 	};
 
@@ -259,7 +270,7 @@ function VariableForm({
 										label: SORT_LABEL[sort],
 										value: sort,
 									}))}
-									onChange={(value): void => set({ sort: value as VariableSort })}
+									onChange={(value): void => set({ sort: value as VariableSortDTO })}
 									testId="variable-sort-select"
 								/>
 							</div>

frontend/src/pages/DashboardPageV2/DashboardContainer/DashboardSettings/Variables/variableAdapters.ts

@@ -1,16 +1,17 @@
 import {
-	DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpecDTOKind as TextEnvelopeKind,
+	DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpecDTOKind as TextEnvelopeKind,
 	DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpecDTOKind as ListEnvelopeKind,
 	DashboardtypesVariablePluginVariantGithubComSigNozSignozPkgTypesDashboardtypesCustomVariableSpecDTOKind as CustomPluginKind,
 	DashboardtypesVariablePluginVariantGithubComSigNozSignozPkgTypesDashboardtypesDynamicVariableSpecDTOKind as DynamicPluginKind,
 	DashboardtypesVariablePluginVariantGithubComSigNozSignozPkgTypesDashboardtypesQueryVariableSpecDTOKind as QueryPluginKind,
+	DashboardtypesListVariableSpecSortDTO as VariableSortDTO,
 	TelemetrytypesSignalDTO,
 } from 'api/generated/services/sigNoz.schemas';
 import type {
 	DashboardtypesListVariableSpecDTO,
 	DashboardtypesVariableDTO,
 	DashboardtypesVariablePluginDTO,
-	DashboardTextVariableSpecDTO,
+	DashboardtypesTextVariableSpecDTO,
 } from 'api/generated/services/sigNoz.schemas';
 
 import {
@@ -18,7 +19,6 @@ import {
 	PLUGIN_KIND,
 	type TelemetrySignal,
 	type VariableFormModel,
-	type VariableSort,
 } from './variableModel';
 
 /** DTO envelope → flat form model (for display / editing). */
@@ -35,7 +35,7 @@ export function dtoToFormModel(
 
 	// Text variable — a distinct envelope (no list plugin).
 	if (dto.kind === TextEnvelopeKind.TextVariable) {
-		const spec = dto.spec as DashboardTextVariableSpecDTO;
+		const spec = dto.spec as DashboardtypesTextVariableSpecDTO;
 		return {
 			...common,
 			type: 'TEXT',
@@ -50,7 +50,7 @@ export function dtoToFormModel(
 		...common,
 		multiSelect: spec.allowMultiple ?? false,
 		showAllOption: spec.allowAllValue ?? false,
-		sort: (spec.sort as VariableSort) ?? 'DISABLED',
+		sort: spec.sort ?? VariableSortDTO.none,
 		defaultValue: spec.defaultValue,
 	};
 	const plugin = spec.plugin;

frontend/src/pages/DashboardPageV2/DashboardContainer/DashboardSettings/Variables/variableModel.ts

@@ -1,4 +1,6 @@
+import { DashboardtypesListVariableSpecSortDTO as VariableSortDTO } from 'api/generated/services/sigNoz.schemas';
 import type { VariableDefaultValueDTO } from 'api/generated/services/sigNoz.schemas';
+import type { TSortVariableValuesType } from 'types/api/dashboard/getAll';
 
 /**
  * Flat, UI-friendly representation of a V2 dashboard variable. The wire format
@@ -8,8 +10,6 @@ import type { VariableDefaultValueDTO } from 'api/generated/services/sigNoz.sche
 
 export type VariableType = 'QUERY' | 'CUSTOM' | 'TEXT' | 'DYNAMIC';
 
-export type VariableSort = 'DISABLED' | 'ASC' | 'DESC';
-
 export type TelemetrySignal = 'traces' | 'logs' | 'metrics';
 
 /** Wire `kind` discriminators (string values of the generated enums). */
@@ -24,7 +24,20 @@ export const PLUGIN_KIND = {
 	DYNAMIC: 'signoz/DynamicVariable',
 } as const;
 
-export const VARIABLE_SORTS: VariableSort[] = ['DISABLED', 'ASC', 'DESC'];
+export const VARIABLE_SORTS: VariableSortDTO[] = Object.values(VariableSortDTO);
+
+/** Direction the preview sorter should apply for a given wire sort value. */
+export function sortDirectionOf(
+	sort: VariableSortDTO,
+): TSortVariableValuesType {
+	if (sort.endsWith('-asc')) {
+		return 'ASC';
+	}
+	if (sort.endsWith('-desc')) {
+		return 'DESC';
+	}
+	return 'DISABLED';
+}
 
 export const TELEMETRY_SIGNALS: TelemetrySignal[] = [
 	'traces',
@@ -42,7 +55,7 @@ export interface VariableFormModel {
 	// List-variable common fields (Query / Custom / Dynamic).
 	multiSelect: boolean;
 	showAllOption: boolean;
-	sort: VariableSort;
+	sort: VariableSortDTO;
 
 	// Type-specific.
 	queryValue: string; // QUERY
@@ -67,7 +80,7 @@ export function emptyVariableFormModel(): VariableFormModel {
 		type: 'QUERY',
 		multiSelect: false,
 		showAllOption: false,
-		sort: 'DISABLED',
+		sort: VariableSortDTO.none,
 		queryValue: '',
 		customValue: '',
 		textValue: '',

pkg/types/dashboardtypes/perses_dashboard_convertors_test.go

@@ -38,7 +38,7 @@ func newTestDashboardV2(t *testing.T, orgID valuer.UUID, source Source) *Dashboa
 								FillMode:          FillModeSolid,
 								SpanGaps:          SpanGaps{FillLessThan: valuer.MustParseTextDuration("60s")},
 							},
-							Legend: Legend{Position: LegendPositionBottom},
+							Legend: Legend{Position: LegendPositionBottom, Mode: LegendModeList},
 						},
 					},
 					Queries: []Query{

pkg/types/dashboardtypes/perses_dashboard_data.go

@@ -48,7 +48,42 @@ func (d *DashboardSpec) UnmarshalJSON(data []byte) error {
 // ══════════════════════════════════════════════
 
 func (d *DashboardSpec) Validate() error {
+	if err := d.validateVariables(); err != nil {
+		return err
+	}
+	if err := d.validatePanels(); err != nil {
+		return err
+	}
+	return d.validateLayouts()
+}
+
+// validateVariables rejects two variables sharing the same name.
+func (d *DashboardSpec) validateVariables() error {
+	seen := make(map[string]struct{}, len(d.Variables))
+	for i, v := range d.Variables {
+		var name string
+		switch s := v.Spec.(type) {
+		case *ListVariableSpec:
+			name = s.Name
+		case *TextVariableSpec:
+			name = s.Name
+		default:
+			// Unreachable via UnmarshalJSON; reaching here means a Go caller broke the Kind/Spec pairing.
+			return errors.NewInternalf(errors.CodeInternal, "spec.variables[%d].spec: unexpected variable spec type %T", i, v.Spec)
+		}
+		if _, dup := seen[name]; dup {
+			return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "spec.variables[%d]: duplicate variable name %q", i, name)
+		}
+		seen[name] = struct{}{}
+	}
+	return nil
+}
+
+func (d *DashboardSpec) validatePanels() error {
 	for key, panel := range d.Panels {
+		if err := common.ValidateID(key); err != nil {
+			return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "spec.panels: %s", err.Error())
+		}
 		if panel == nil {
 			return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "spec.panels.%s: panel must not be null", key)
 		}
@@ -69,6 +104,13 @@ func (d *DashboardSpec) Validate() error {
 }
 
 func validateQueryAllowedForPanel(plugin QueryPlugin, allowed []QueryPluginKind, panelKind PanelPluginKind, path string) error {
+	compositeSubQueryTypeToPluginKind := map[qb.QueryType]QueryPluginKind{
+		qb.QueryTypeBuilder:       QueryKindBuilder,
+		qb.QueryTypeFormula:       QueryKindFormula,
+		qb.QueryTypeTraceOperator: QueryKindTraceOperator,
+		qb.QueryTypePromQL:        QueryKindPromQL,
+		qb.QueryTypeClickHouseSQL: QueryKindClickHouseSQL,
+	}
 	if !slices.Contains(allowed, plugin.Kind) {
 		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput,
 			"%s: query kind %q is not supported by panel kind %q", path, plugin.Kind, panelKind)
@@ -96,12 +138,35 @@ func validateQueryAllowedForPanel(plugin QueryPlugin, allowed []QueryPluginKind,
 	return nil
 }
 
-var (
-	compositeSubQueryTypeToPluginKind = map[qb.QueryType]QueryPluginKind{
-		qb.QueryTypeBuilder:       QueryKindBuilder,
-		qb.QueryTypeFormula:       QueryKindFormula,
-		qb.QueryTypeTraceOperator: QueryKindTraceOperator,
-		qb.QueryTypePromQL:        QueryKindPromQL,
-		qb.QueryTypeClickHouseSQL: QueryKindClickHouseSQL,
+// validateLayouts rejects grid items referencing a panel that doesn't exist.
+func (d *DashboardSpec) validateLayouts() error {
+	for li, layout := range d.Layouts {
+		grid, ok := layout.Spec.(*dashboard.GridLayoutSpec)
+		if !ok {
+			// Unreachable via UnmarshalJSON; reaching here means a Go caller broke the Kind/Spec pairing.
+			return errors.NewInternalf(errors.CodeInternal, "spec.layouts[%d].spec: unexpected layout spec type %T", li, layout.Spec)
+		}
+		for ii, item := range grid.Items {
+			path := fmt.Sprintf("spec.layouts[%d].spec.items[%d].content", li, ii)
+			if item.Content == nil {
+				return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s: content reference is required", path)
+			}
+			key, err := panelKeyFromRef(item.Content.Path, item.Content.Ref, path)
+			if err != nil {
+				return err
+			}
+			if _, ok := d.Panels[key]; !ok {
+				return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s: references unknown panel %q", path, key)
+			}
+		}
 	}
-)
+	return nil
+}
+
+// panelKeyFromRef extracts <key> from a "#/spec/panels/<key>" content ref.
+func panelKeyFromRef(refPath []string, ref string, path string) (string, error) {
+	if len(refPath) != 3 || refPath[0] != "spec" || refPath[1] != "panels" {
+		return "", errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "%s: %q must reference a panel as \"#/spec/panels/<key>\"", path, ref)
+	}
+	return refPath[2], nil
+}

pkg/types/dashboardtypes/perses_dashboard_patch.go

@@ -73,7 +73,7 @@ func (p PatchableDashboardV2) Apply(existing *DashboardV2) (*UpdatableDashboardV
 	}
 	patched, err := p.patch.ApplyWithOptions(raw, &jsonpatch.ApplyOptions{AllowMissingPathOnRemove: true, EnsurePathExistsOnAdd: true})
 	if err != nil {
-		return nil, errors.Wrap(err, errors.TypeInvalidInput, ErrCodeDashboardInvalidPatch, "JSON Patch could not be applied to the target dashboard")
+		return nil, errors.Wrap(err, errors.TypeInvalidInput, ErrCodeDashboardInvalidPatch, "JSON Patch could not be applied to the target dashboard").WithAdditional(err.Error())
 	}
 	out := &UpdatableDashboardV2{}
 	if err := json.Unmarshal(patched, out); err != nil {

pkg/types/dashboardtypes/perses_dashboard_patch_test.go

@@ -405,6 +405,7 @@ func TestPatchableDashboardV2_Apply(t *testing.T) {
 		out, err := decode(t, `[
 			{"op": "replace", "path": "/spec/display/name", "value": "Multi-step"},
 			{"op": "remove",  "path": "/spec/panels/p2"},
+			{"op": "remove",  "path": "/spec/layouts/0/spec/items/1"},
 			{"op": "add",     "path": "/tags/-", "value": {"key": "env", "value": "staging"}}
 		]`).Apply(base)
 		require.NoError(t, err)

pkg/types/dashboardtypes/perses_dashboard_test.go

@@ -112,6 +112,174 @@ func TestValidateOnlyVariables(t *testing.T) {
 	require.NoError(t, err, "expected valid")
 }
 
+func TestInvalidateDuplicateVariableNames(t *testing.T) {
+	data := []byte(`{
+		"variables": [
+			{
+				"kind": "TextVariable",
+				"spec": {"name": "env", "value": "prod"}
+			},
+			{
+				"kind": "ListVariable",
+				"spec": {
+					"name": "env",
+					"allowAllValue": false,
+					"allowMultiple": false,
+					"plugin": {
+						"kind": "signoz/DynamicVariable",
+						"spec": {"name": "service.name", "signal": "metrics"}
+					}
+				}
+			}
+		],
+		"layouts": []
+	}`)
+	_, err := unmarshalDashboard(data)
+	require.Error(t, err, "expected error for duplicate variable name")
+	require.Contains(t, err.Error(), `duplicate variable name "env"`)
+}
+
+func TestInvalidateVariableNameWithInvalidChars(t *testing.T) {
+	listVarWithName := func(name string) []byte {
+		return []byte(`{
+			"variables": [
+				{
+					"kind": "ListVariable",
+					"spec": {
+						"name": "` + name + `",
+						"allowAllValue": false,
+						"allowMultiple": false,
+						"plugin": {
+							"kind": "signoz/DynamicVariable",
+							"spec": {"name": "service.name", "signal": "metrics"}
+						}
+					}
+				}
+			],
+			"layouts": []
+		}`)
+	}
+	for _, name := range []string{"my var", "cost$", "bad!", "a/b"} {
+		t.Run(name, func(t *testing.T) {
+			_, err := unmarshalDashboard(listVarWithName(name))
+			require.Error(t, err, "expected error for invalid variable name %q", name)
+			require.Contains(t, err.Error(), "is not a correct name")
+		})
+	}
+	for _, name := range []string{"service", "my_var", "MY_VAR", "MixedCase9", "with-hyphen", "with.dot"} {
+		t.Run(name, func(t *testing.T) {
+			_, err := unmarshalDashboard(listVarWithName(name))
+			require.NoError(t, err, "expected valid variable name %q", name)
+		})
+	}
+	t.Run("digits only", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVarWithName("123"))
+		require.Error(t, err)
+		require.Contains(t, err.Error(), "cannot contain only digits")
+	})
+}
+
+func TestInvalidatePanelKey(t *testing.T) {
+	data := []byte(`{
+		"panels": {
+			"bad key!": {
+				"kind": "Panel",
+				"spec": {
+					"plugin": {"kind": "signoz/TablePanel", "spec": {}},
+					"queries": [{
+						"kind": "time_series",
+						"spec": {"plugin": {"kind": "signoz/BuilderQuery", "spec": {
+							"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}]
+						}}}
+					}]
+				}
+			}
+		},
+		"layouts": []
+	}`)
+	_, err := unmarshalDashboard(data)
+	require.Error(t, err, "expected error for invalid panel key")
+	require.Contains(t, err.Error(), "is not a correct name")
+}
+
+func TestInvalidateListVariableCrossFields(t *testing.T) {
+	listVar := func(specFields string) []byte {
+		return []byte(`{
+			"variables": [
+				{
+					"kind": "ListVariable",
+					"spec": {
+						"name": "service",
+						` + specFields + `
+						"plugin": {
+							"kind": "signoz/DynamicVariable",
+							"spec": {"name": "service.name", "signal": "metrics"}
+						}
+					}
+				}
+			],
+			"layouts": []
+		}`)
+	}
+
+	t.Run("customAllValue without allowAllValue", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVar(`"allowAllValue": false, "allowMultiple": false, "customAllValue": "*",`))
+		require.Error(t, err)
+		require.Contains(t, err.Error(), "customAllValue cannot be set")
+	})
+
+	t.Run("list defaultValue without allowMultiple", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVar(`"allowAllValue": false, "allowMultiple": false, "defaultValue": ["a", "b"],`))
+		require.Error(t, err)
+		require.Contains(t, err.Error(), "allowMultiple")
+	})
+
+	t.Run("single-element list default without allowMultiple", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVar(`"allowAllValue": false, "allowMultiple": false, "defaultValue": ["only"],`))
+		require.Error(t, err)
+		require.Contains(t, err.Error(), "allowMultiple")
+	})
+
+	t.Run("valid sort is accepted", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVar(`"sort": "alphabetical-asc",`))
+		require.NoError(t, err)
+	})
+
+	t.Run("unknown sort is rejected", func(t *testing.T) {
+		_, err := unmarshalDashboard(listVar(`"sort": "bogus",`))
+		require.Error(t, err)
+		require.Contains(t, err.Error(), "unknown sort")
+	})
+}
+
+func TestInvalidateEmptyVariableName(t *testing.T) {
+	cases := map[string][]byte{
+		"text variable": []byte(`{
+			"variables": [{"kind": "TextVariable", "spec": {"name": "", "value": "x"}}],
+			"layouts": []
+		}`),
+		"list variable": []byte(`{
+			"variables": [{
+				"kind": "ListVariable",
+				"spec": {
+					"name": "",
+					"allowAllValue": false,
+					"allowMultiple": false,
+					"plugin": {"kind": "signoz/DynamicVariable", "spec": {"name": "service.name", "signal": "metrics"}}
+				}
+			}],
+			"layouts": []
+		}`),
+	}
+	for name, data := range cases {
+		t.Run(name, func(t *testing.T) {
+			_, err := unmarshalDashboard(data)
+			require.Error(t, err, "expected error for empty variable name")
+			require.Contains(t, err.Error(), "name cannot be empty")
+		})
+	}
+}
+
 func TestInvalidateUnknownPluginKind(t *testing.T) {
 	tests := []struct {
 		name        string
@@ -270,6 +438,65 @@ func TestInvalidateOneInvalidPanel(t *testing.T) {
 	require.Contains(t, err.Error(), "FakePanel", "error should mention FakePanel")
 }
 
+func TestInvalidateLayoutPanelReferences(t *testing.T) {
+	validPanels := `"panels": {
+		"p1": {
+			"kind": "Panel",
+			"spec": {
+				"plugin": {"kind": "signoz/TablePanel", "spec": {}},
+				"queries": [{
+					"kind": "time_series",
+					"spec": {"plugin": {"kind": "signoz/BuilderQuery", "spec": {
+						"name": "A", "signal": "logs", "aggregations": [{"expression": "count()"}]
+					}}}
+				}]
+			}
+		}
+	}`
+	layout := func(items string) []byte {
+		return []byte(`{` + validPanels + `, "layouts": [{"kind": "Grid", "spec": {"items": [` + items + `]}}]}`)
+	}
+
+	tests := []struct {
+		name        string
+		data        []byte
+		wantContain string
+	}{
+		{
+			name:        "reference to unknown panel",
+			data:        layout(`{"x": 0, "y": 0, "width": 6, "height": 6, "content": {"$ref": "#/spec/panels/ghost"}}`),
+			wantContain: `references unknown panel "ghost"`,
+		},
+		{
+			name:        "reference not pointing at a panel",
+			data:        layout(`{"x": 0, "y": 0, "width": 6, "height": 6, "content": {"$ref": "#/spec/variables/p1"}}`),
+			wantContain: "must reference a panel",
+		},
+		{
+			name:        "reference missing spec prefix",
+			data:        layout(`{"x": 0, "y": 0, "width": 6, "height": 6, "content": {"$ref": "#/panels/p1"}}`),
+			wantContain: "must reference a panel",
+		},
+		{
+			name:        "valid reference",
+			data:        layout(`{"x": 0, "y": 0, "width": 6, "height": 6, "content": {"$ref": "#/spec/panels/p1"}}`),
+			wantContain: "",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_, err := unmarshalDashboard(tt.data)
+			if tt.wantContain == "" {
+				require.NoError(t, err)
+				return
+			}
+			require.Error(t, err)
+			require.Contains(t, err.Error(), tt.wantContain)
+		})
+	}
+}
+
 func TestRejectUnknownFieldsInPluginSpec(t *testing.T) {
 	tests := []struct {
 		name        string
@@ -569,6 +796,24 @@ func TestInvalidateBadPanelSpecValues(t *testing.T) {
 			}`,
 			wantContain: "legend position",
 		},
+		{
+			name: "bad legend mode",
+			data: `{
+				"panels": {
+					"p1": {
+						"kind": "Panel",
+						"spec": {
+							"plugin": {
+								"kind": "signoz/BarChartPanel",
+								"spec": {"legend": {"mode": "grid"}}
+							}
+						}
+					}
+				},
+				"layouts": []
+			}`,
+			wantContain: "legend mode",
+		},
 		{
 			name: "bad threshold format",
 			data: `{
@@ -634,6 +879,39 @@ func TestInvalidateBadPanelSpecValues(t *testing.T) {
 	}
 }
 
+// Label on ThresholdWithLabel is optional — the backend never reads it, so a
+// threshold with an omitted or empty label must validate cleanly.
+func TestThresholdLabelOptional(t *testing.T) {
+	for _, tt := range []struct {
+		name      string
+		threshold string
+	}{
+		{name: "label omitted", threshold: `{"value": 100, "color": "Red"}`},
+		{name: "label empty", threshold: `{"value": 100, "color": "Red", "label": ""}`},
+	} {
+		t.Run(tt.name, func(t *testing.T) {
+			data := []byte(`{
+				"panels": {
+					"p1": {
+						"kind": "Panel",
+						"spec": {
+							"plugin": {"kind": "signoz/TimeSeriesPanel", "spec": {"thresholds": [` + tt.threshold + `]}},
+							"queries": [{"kind": "time_series", "spec": {"plugin": {"kind": "signoz/PromQLQuery", "spec": {"name": "A", "query": "up"}}}}]
+						}
+					}
+				},
+				"layouts": []
+			}`)
+			d, err := unmarshalDashboard(data)
+			require.NoError(t, err, "threshold without a label should validate")
+
+			spec := d.Panels["p1"].Spec.Plugin.Spec.(*TimeSeriesPanelSpec)
+			require.Len(t, spec.Thresholds, 1)
+			require.Empty(t, spec.Thresholds[0].Label, "label should remain empty")
+		})
+	}
+}
+
 func TestInvalidatePanelWithoutQueries(t *testing.T) {
 	data := []byte(`{
 		"panels": {
@@ -749,11 +1027,6 @@ func TestValidateRequiredFields(t *testing.T) {
 			data:        wrapPanel("signoz/TimeSeriesPanel", `{"thresholds": [{"value": 100, "label": "high", "color": ""}]}`),
 			wantContain: "Color",
 		},
-		{
-			name:        "ThresholdWithLabel missing label",
-			data:        wrapPanel("signoz/TimeSeriesPanel", `{"thresholds": [{"value": 100, "color": "Red", "label": ""}]}`),
-			wantContain: "Label",
-		},
 		{
 			name:        "ComparisonThreshold missing value",
 			data:        wrapPanel("signoz/NumberPanel", `{"thresholds": [{"operator": "above", "format": "text", "color": "Red"}]}`),
@@ -811,10 +1084,11 @@ func TestTimeSeriesPanelDefaults(t *testing.T) {
 	require.Equal(t, "2", spec.Formatting.DecimalPrecision.ValueOrDefault(), "expected DecimalPrecision default 2")
 	require.Equal(t, "spline", spec.ChartAppearance.LineInterpolation.ValueOrDefault(), "expected LineInterpolation default spline")
 	require.Equal(t, "solid", spec.ChartAppearance.LineStyle.ValueOrDefault(), "expected LineStyle default solid")
-	require.Equal(t, "solid", spec.ChartAppearance.FillMode.ValueOrDefault(), "expected FillMode default solid")
+	require.Equal(t, "none", spec.ChartAppearance.FillMode.ValueOrDefault(), "expected FillMode default none")
 	require.False(t, spec.ChartAppearance.SpanGaps.FillOnlyBelow, "expected SpanGaps.FillOnlyBelow default false")
 	require.Equal(t, "global_time", spec.Visualization.TimePreference.ValueOrDefault(), "expected TimePreference default global_time")
 	require.Equal(t, "bottom", spec.Legend.Position.ValueOrDefault(), "expected LegendPosition default bottom")
+	require.Equal(t, "list", spec.Legend.Mode.ValueOrDefault(), "expected LegendMode default list")
 
 	// Re-marshal the full dashboard (what we'd store in DB / return in API response)
 	// and verify the output contains the default values.
@@ -825,9 +1099,10 @@ func TestTimeSeriesPanelDefaults(t *testing.T) {
 		"decimalPrecision":  `"2"`,
 		"lineInterpolation": `"spline"`,
 		"lineStyle":         `"solid"`,
-		"fillMode":          `"solid"`,
+		"fillMode":          `"none"`,
 		"timePreference":    `"global_time"`,
 		"position":          `"bottom"`,
+		"mode":              `"list"`,
 	} {
 		assert.Contains(t, outputStr, `"`+field+`":`+want, "expected stored/response JSON to contain %s:%s", field, want)
 	}
@@ -930,7 +1205,7 @@ func TestStorageRoundTrip(t *testing.T) {
 	assert.Equal(t, "2", tsSpec.Formatting.DecimalPrecision.ValueOrDefault())
 	assert.Equal(t, "spline", tsSpec.ChartAppearance.LineInterpolation.ValueOrDefault())
 	assert.Equal(t, "solid", tsSpec.ChartAppearance.LineStyle.ValueOrDefault())
-	assert.Equal(t, "solid", tsSpec.ChartAppearance.FillMode.ValueOrDefault())
+	assert.Equal(t, "none", tsSpec.ChartAppearance.FillMode.ValueOrDefault())
 	assert.Equal(t, "global_time", tsSpec.Visualization.TimePreference.ValueOrDefault())
 	assert.Equal(t, "bottom", tsSpec.Legend.Position.ValueOrDefault())
 	numSpec := d.Panels["p2"].Spec.Plugin.Spec.(*NumberPanelSpec)
@@ -950,7 +1225,7 @@ func TestStorageRoundTrip(t *testing.T) {
 	assert.Equal(t, "2", tsLoaded.Formatting.DecimalPrecision.ValueOrDefault(), "after load")
 	assert.Equal(t, "spline", tsLoaded.ChartAppearance.LineInterpolation.ValueOrDefault(), "after load")
 	assert.Equal(t, "solid", tsLoaded.ChartAppearance.LineStyle.ValueOrDefault(), "after load")
-	assert.Equal(t, "solid", tsLoaded.ChartAppearance.FillMode.ValueOrDefault(), "after load")
+	assert.Equal(t, "none", tsLoaded.ChartAppearance.FillMode.ValueOrDefault(), "after load")
 	assert.Equal(t, "global_time", tsLoaded.Visualization.TimePreference.ValueOrDefault(), "after load")
 	assert.Equal(t, "bottom", tsLoaded.Legend.Position.ValueOrDefault(), "after load")
 	numLoaded := loaded.Panels["p2"].Spec.Plugin.Spec.(*NumberPanelSpec)
@@ -966,7 +1241,7 @@ func TestStorageRoundTrip(t *testing.T) {
 		"decimalPrecision":  `"2"`,
 		"lineInterpolation": `"spline"`,
 		"lineStyle":         `"solid"`,
-		"fillMode":          `"solid"`,
+		"fillMode":          `"none"`,
 		"timePreference":    `"global_time"`,
 		"position":          `"bottom"`,
 		"format":            `"text"`,

pkg/types/dashboardtypes/perses_drift_test.go

@@ -30,6 +30,7 @@ func TestDashboardSpecMatchesPerses(t *testing.T) {
 		{"DatasourceSpec", typeOf[DatasourceSpec](), typeOf[datasource.Spec]()},
 		{"Variable", typeOf[Variable](), typeOf[dashboard.Variable]()},
 		{"ListVariableSpec", typeOf[ListVariableSpec](), typeOf[dashboard.ListVariableSpec]()},
+		{"TextVariableSpec", typeOf[TextVariableSpec](), typeOf[dashboard.TextVariableSpec]()},
 		{"Layout", typeOf[Layout](), typeOf[dashboard.Layout]()},
 	}
 

pkg/types/dashboardtypes/perses_plugin_wrappers.go

@@ -51,7 +51,7 @@ func (p *PanelPlugin) UnmarshalJSON(data []byte) error {
 		return err
 	}
 	p.Kind = PanelPluginKind(kind)
-	p.Spec = spec
+	p.Spec = *spec
 	return nil
 }
 
@@ -110,7 +110,7 @@ func (p *QueryPlugin) UnmarshalJSON(data []byte) error {
 		return err
 	}
 	p.Kind = QueryPluginKind(kind)
-	p.Spec = spec
+	p.Spec = *spec
 	return nil
 }
 
@@ -165,7 +165,7 @@ func (p *VariablePlugin) UnmarshalJSON(data []byte) error {
 		return err
 	}
 	p.Kind = VariablePluginKind(kind)
-	p.Spec = spec
+	p.Spec = *spec
 	return nil
 }
 
@@ -215,7 +215,7 @@ func (p *DatasourcePlugin) UnmarshalJSON(data []byte) error {
 		return err
 	}
 	p.Kind = DatasourcePluginKind(kind)
-	p.Spec = spec
+	p.Spec = *spec
 	return nil
 }
 
@@ -297,8 +297,7 @@ func extractKindAndSpec(data []byte) (string, []byte, error) {
 	return head.Kind, head.Spec, nil
 }
 
-// decodeSpec strict-decodes a spec JSON into target and runs struct-tag validation (go-playground/validator).
-func decodeSpec(specJSON []byte, target any, kind string) (any, error) {
+func decodeSpec[T any](specJSON []byte, target T, kind string) (*T, error) {
 	if len(specJSON) == 0 {
 		return nil, errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "kind %q: spec is required", kind)
 	}
@@ -310,7 +309,12 @@ func decodeSpec(specJSON []byte, target any, kind string) (any, error) {
 	if err := validator.New().Struct(target); err != nil {
 		return nil, errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "kind %q: spec failed validation", kind)
 	}
-	return target, nil
+	if v, ok := any(target).(interface{ validate() error }); ok {
+		if err := v.validate(); err != nil {
+			return nil, errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "kind %q: %s", kind, err.Error())
+		}
+	}
+	return &target, nil
 }
 
 // signozDiscriminatorKey is the extension key that signoz.attachDiscriminators

pkg/types/dashboardtypes/perses_replicas.go

@@ -4,9 +4,11 @@ import (
 	"encoding/json"
 	"maps"
 	"slices"
+	"strconv"
 
 	"github.com/SigNoz/signoz/pkg/errors"
 	qb "github.com/SigNoz/signoz/pkg/types/querybuildertypes/querybuildertypesv5"
+	"github.com/SigNoz/signoz/pkg/valuer"
 	"github.com/perses/spec/go/common"
 	"github.com/perses/spec/go/dashboard"
 	"github.com/perses/spec/go/dashboard/variable"
@@ -84,7 +86,7 @@ type QuerySpec struct {
 // ══════════════════════════════════════════════
 
 // Variable is the list/text sum type. Spec is set to *ListVariableSpec or
-// *dashboard.TextVariableSpec by UnmarshalJSON based on Kind. The schema is a
+// *TextVariableSpec by UnmarshalJSON based on Kind. The schema is a
 // discriminated oneOf (see JSONSchemaOneOf).
 type Variable struct {
 	Kind variable.Kind `json:"kind" required:"true"`
@@ -94,7 +96,7 @@ type Variable struct {
 func (Variable) PrepareJSONSchema(s *jsonschema.Schema) error {
 	return markDiscriminator(s, "kind", map[string]string{
 		string(variable.KindList): schemaRef("DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesListVariableSpec"),
-		string(variable.KindText): schemaRef("DashboardtypesVariableEnvelopeGithubComPersesSpecGoDashboardTextVariableSpec"),
+		string(variable.KindText): schemaRef("DashboardtypesVariableEnvelopeGithubComSigNozSignozPkgTypesDashboardtypesTextVariableSpec"),
 	})
 }
 
@@ -110,14 +112,14 @@ func (v *Variable) UnmarshalJSON(data []byte) error {
 			return err
 		}
 		v.Kind = variable.KindList
-		v.Spec = spec
+		v.Spec = *spec
 	case string(variable.KindText):
-		spec, err := decodeSpec(specJSON, new(dashboard.TextVariableSpec), kind)
+		spec, err := decodeSpec(specJSON, new(TextVariableSpec), kind)
 		if err != nil {
 			return err
 		}
 		v.Kind = variable.KindText
-		v.Spec = spec
+		v.Spec = *spec
 	default:
 		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "unknown variable kind %q; allowed values: %s", kind, allowedValuesForKind([]variable.Kind{variable.KindList, variable.KindText}))
 	}
@@ -127,7 +129,7 @@ func (v *Variable) UnmarshalJSON(data []byte) error {
 func (Variable) JSONSchemaOneOf() []any {
 	return []any{
 		VariableEnvelope[ListVariableSpec]{Kind: string(variable.KindList)},
-		VariableEnvelope[dashboard.TextVariableSpec]{Kind: string(variable.KindText)},
+		VariableEnvelope[TextVariableSpec]{Kind: string(variable.KindText)},
 	}
 }
 
@@ -143,15 +145,106 @@ func (v VariableEnvelope[S]) PrepareJSONSchema(s *jsonschema.Schema) error {
 // ListVariableSpec mirrors dashboard.ListVariableSpec (variable.ListSpec
 // fields + Name) but with a typed VariablePlugin replacing common.Plugin.
 type ListVariableSpec struct {
-	Display         Display                `json:"display" required:"true"`
+	Display         *Display               `json:"display,omitempty"`
 	DefaultValue    *variable.DefaultValue `json:"defaultValue,omitempty"`
 	AllowAllValue   bool                   `json:"allowAllValue"`
 	AllowMultiple   bool                   `json:"allowMultiple"`
 	CustomAllValue  string                 `json:"customAllValue,omitempty"`
 	CapturingRegexp string                 `json:"capturingRegexp,omitempty"`
-	Sort            *variable.Sort         `json:"sort,omitempty"`
+	Sort            ListVariableSpecSort   `json:"sort,omitzero"`
 	Plugin          VariablePlugin         `json:"plugin"`
-	Name            string                 `json:"name"`
+	Name            string                 `json:"name" required:"true" minLength:"1"`
+}
+
+// validate mirrors perses ListVariableSpec validation (plus the digits-only name
+// check perses only applies to text variables); run by decodeSpec on unmarshal.
+func (s *ListVariableSpec) validate() error {
+	if err := common.ValidateID(s.Name); err != nil {
+		return err
+	}
+	if _, err := strconv.Atoi(s.Name); err == nil {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "variable name cannot contain only digits")
+	}
+	if s.CustomAllValue != "" && !s.AllowAllValue {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "customAllValue cannot be set if allowAllValue is not set to true")
+	}
+	if s.DefaultValue != nil && len(s.DefaultValue.SliceValues) > 0 && !s.AllowMultiple {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "defaultValue cannot be a list if allowMultiple is not set to true")
+	}
+	return nil
+}
+
+// ListVariableSpecSort is the value-list sort method, mirrored from Perses as a
+// stable enum so the allowed values surface in the generated OpenAPI schema.
+type ListVariableSpecSort struct{ valuer.String }
+
+var (
+	SortNone                            = ListVariableSpecSort{valuer.NewString("none")}
+	SortAlphabeticalAsc                 = ListVariableSpecSort{valuer.NewString("alphabetical-asc")}
+	SortAlphabeticalDesc                = ListVariableSpecSort{valuer.NewString("alphabetical-desc")}
+	SortNumericalAsc                    = ListVariableSpecSort{valuer.NewString("numerical-asc")}
+	SortNumericalDesc                   = ListVariableSpecSort{valuer.NewString("numerical-desc")}
+	SortAlphabeticalCaseInsensitiveAsc  = ListVariableSpecSort{valuer.NewString("alphabetical-ci-asc")}
+	SortAlphabeticalCaseInsensitiveDesc = ListVariableSpecSort{valuer.NewString("alphabetical-ci-desc")}
+)
+
+func (ListVariableSpecSort) Enum() []any {
+	return []any{
+		SortNone,
+		SortAlphabeticalAsc,
+		SortAlphabeticalDesc,
+		SortNumericalAsc,
+		SortNumericalDesc,
+		SortAlphabeticalCaseInsensitiveAsc,
+		SortAlphabeticalCaseInsensitiveDesc,
+	}
+}
+
+func (s ListVariableSpecSort) IsValid() bool {
+	return slices.ContainsFunc(s.Enum(), func(v any) bool { return v == s })
+}
+
+// UnmarshalJSON validates against the enum on decode (valuer.String alone
+// accepts any string). An empty value is allowed and means "no sort", matching
+// Perses.
+func (s *ListVariableSpecSort) UnmarshalJSON(data []byte) error {
+	var v string
+	if err := json.Unmarshal(data, &v); err != nil {
+		return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid sort: must be a string, one of `none`, `alphabetical-asc`, `alphabetical-desc`, `numerical-asc`, `numerical-desc`, `alphabetical-ci-asc`, or `alphabetical-ci-desc`")
+	}
+	if v == "" {
+		*s = ListVariableSpecSort{}
+		return nil
+	}
+	sort := ListVariableSpecSort{valuer.NewString(v)}
+	if !sort.IsValid() {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "unknown sort %q: must be `none`, `alphabetical-asc`, `alphabetical-desc`, `numerical-asc`, `numerical-desc`, `alphabetical-ci-asc`, or `alphabetical-ci-desc`", v)
+	}
+	*s = sort
+	return nil
+}
+
+// TextVariableSpec replicates dashboard.TextVariableSpec so name can carry the
+// required/non-empty schema tags perses leaves off.
+type TextVariableSpec struct {
+	Display  *Display `json:"display,omitempty"`
+	Value    string   `json:"value"`
+	Constant bool     `json:"constant,omitempty"`
+	Name     string   `json:"name" required:"true" minLength:"1"`
+}
+
+// validate mirrors perses TextVariableSpec validation; run by decodeSpec on unmarshal.
+func (s *TextVariableSpec) validate() error {
+	if err := common.ValidateID(s.Name); err != nil {
+		return err
+	}
+	if _, err := strconv.Atoi(s.Name); err == nil {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "variable name cannot contain only digits")
+	}
+	if s.Value == "" && s.Constant {
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "value for a constant text variable cannot be empty")
+	}
+	return nil
 }
 
 // ══════════════════════════════════════════════
@@ -194,7 +287,7 @@ func (l *Layout) UnmarshalJSON(data []byte) error {
 		return err
 	}
 	l.Kind = dashboard.LayoutKind(kind)
-	l.Spec = spec
+	l.Spec = *spec
 	return nil
 }
 

pkg/types/dashboardtypes/perses_signoz_plugins.go

@@ -241,6 +241,7 @@ type TableFormatting struct {
 
 type Legend struct {
 	Position     LegendPosition    `json:"position"`
+	Mode         LegendMode        `json:"mode"`
 	CustomColors map[string]string `json:"customColors"`
 }
 
@@ -248,7 +249,7 @@ type ThresholdWithLabel struct {
 	Value float64 `json:"value" validate:"required" required:"true"`
 	Unit  string  `json:"unit"`
 	Color string  `json:"color" validate:"required" required:"true"`
-	Label string  `json:"label" validate:"required" required:"true"`
+	Label string  `json:"label"`
 }
 
 type ComparisonThreshold struct {
@@ -358,6 +359,47 @@ func (l *LegendPosition) UnmarshalJSON(data []byte) error {
 	}
 }
 
+type LegendMode struct{ valuer.String }
+
+var (
+	LegendModeList  = LegendMode{valuer.NewString("list")} // default
+	LegendModeTable = LegendMode{valuer.NewString("table")}
+)
+
+func (LegendMode) Enum() []any {
+	return []any{LegendModeList} // others are not supported in UI yet
+}
+
+func (m LegendMode) ValueOrDefault() string {
+	if m.IsZero() {
+		return LegendModeList.StringValue()
+	}
+	return m.StringValue()
+}
+
+func (m LegendMode) MarshalJSON() ([]byte, error) {
+	return json.Marshal(m.ValueOrDefault())
+}
+
+func (m *LegendMode) UnmarshalJSON(data []byte) error {
+	var v string
+	if err := json.Unmarshal(data, &v); err != nil {
+		return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid legend mode: must be a string, one of `list` or `table`")
+	}
+	if v == "" {
+		*m = LegendModeList
+		return nil
+	}
+	lm := LegendMode{valuer.NewString(v)}
+	switch lm {
+	case LegendModeList, LegendModeTable:
+		*m = lm
+		return nil
+	default:
+		return errors.NewInvalidInputf(ErrCodeDashboardInvalidInput, "invalid legend mode %q: must be `list` or `table`", v)
+	}
+}
+
 type ThresholdFormat struct{ valuer.String }
 
 var (
@@ -534,9 +576,9 @@ func (ls *LineStyle) UnmarshalJSON(data []byte) error {
 type FillMode struct{ valuer.String }
 
 var (
-	FillModeSolid    = FillMode{valuer.NewString("solid")} // default
+	FillModeSolid    = FillMode{valuer.NewString("solid")}
 	FillModeGradient = FillMode{valuer.NewString("gradient")}
-	FillModeNone     = FillMode{valuer.NewString("none")}
+	FillModeNone     = FillMode{valuer.NewString("none")} // default
 )
 
 func (FillMode) Enum() []any {
@@ -545,7 +587,7 @@ func (FillMode) Enum() []any {
 
 func (fm FillMode) ValueOrDefault() string {
 	if fm.IsZero() {
-		return FillModeSolid.StringValue()
+		return FillModeNone.StringValue()
 	}
 	return fm.StringValue()
 }
@@ -560,7 +602,7 @@ func (fm *FillMode) UnmarshalJSON(data []byte) error {
 		return errors.WrapInvalidInputf(err, ErrCodeDashboardInvalidInput, "invalid fill mode: must be a string, one of `solid`, `gradient`, or `none`")
 	}
 	if v == "" {
-		*fm = FillModeSolid
+		*fm = FillModeNone
 		return nil
 	}
 	val := FillMode{valuer.NewString(v)}
@@ -573,12 +615,9 @@ func (fm *FillMode) UnmarshalJSON(data []byte) error {
 	}
 }
 
-// SpanGaps controls whether lines connect across null values.
-// When FillOnlyBelow is false (default), all gaps are connected.
-// When FillOnlyBelow is true, only gaps smaller than FillLessThan are connected.
 type SpanGaps struct {
-	FillOnlyBelow bool                `json:"fillOnlyBelow"`
-	FillLessThan  valuer.TextDuration `json:"fillLessThan"`
+	FillOnlyBelow bool                `json:"fillOnlyBelow" description:"Controls whether lines connect across null values. When false (default), all gaps are connected. When true, only gaps smaller than fillLessThan are connected."`
+	FillLessThan  valuer.TextDuration `json:"fillLessThan" description:"The maximum gap size to connect when fillOnlyBelow is true. Gaps larger than this duration are left disconnected."`
 }
 
 type PrecisionOption struct{ valuer.String }