Skip to content

Draft: Added sql logs

Michal Zander requested to merge mzander/sql-logs into develop

Firstly we need to create config file promtail-config.yml by creating in specific directory (in my case it is home/dev/docker_volumes/promtail , if you want to change it to another directory you need to change it in docker-compose.yaml aswell to your_directory:/etc/promtail )

---
clients:
  -
    url: "http://loki:3100/loki/api/v1/push"
positions:
  filename: /tmp/positions.yaml
scrape_configs:
  -
    job_name: hive
    pipeline_stages:
      -
        logfmt:
          mapping:
            timestamp:
            user:
            database:
            level:
            context:
            haflog:
    static_configs:
      -
        labels:
          __path__: /var/log/postgresql/*log
          hive: logs
        targets:
          - localhost
  -
    job_name: arch
    static_configs:
      -
        labels:
          __path__: /var/log/postgresql/*log.1
          hive: archive
        targets:
          - localhost
server:
  grpc_listen_port: 0
  http_listen_port: 9080

Next step will be creating our dockers that collect, send and visualize sent logs by creating a docker-compose.yaml file in selected directory

version: "3.8"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.6.0
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  grafana:
    image: grafana/grafana:9.4.3
    ports:
      - "3000:3000"
    networks:
      - loki

  promtail:
    image: grafana/promtail:2.6.0
    volumes:
      - /var/log/:/var/log/
      - /home/dev/docker_volumes/promtail:/etc/promtail
      - /var/log/postgresql/:/var/log/postgresql
    command: -config.file=/etc/promtail/promtail-config.yml
    networks:
      - loki

In postgresql.conf located in /etc/postgresql/12/main (opened with sudo) change log prefixes to

log_line_prefix = 'timestamp="%m" user=%u database=%d level=' 

and log level to highest state

log_min_messages = debug5

After changes in postgresql.conf you need to restart postgres by sudo systemctl restart postgres

Now we can start our dockers by using this command:

docker-compose -f docker-compose.yaml up

in localhost:3000 there is located grafana where we need to setup loki as a source of logs and change url to http://loki:3100

Done.

Edited by Michal Zander

Merge request reports