haf merge requestshttps://gitlab.syncad.com/hive/haf/-/merge_requests2024-02-29T22:05:19Zhttps://gitlab.syncad.com/hive/haf/-/merge_requests/152Draft: Added sql logs2024-02-29T22:05:19ZMichal ZanderDraft: Added sql logsFirstly we need to create config file promtail-config.yml by creating in specific directory (in my case it is ```home/dev/docker_volumes/promtail``` , if you want to change it to another directory you need to change it in docker-compose....Firstly we need to create config file promtail-config.yml by creating in specific directory (in my case it is ```home/dev/docker_volumes/promtail``` , if you want to change it to another directory you need to change it in docker-compose.yaml aswell to ```your_directory:/etc/promtail``` )
```
---
clients:
-
url: "http://loki:3100/loki/api/v1/push"
positions:
filename: /tmp/positions.yaml
scrape_configs:
-
job_name: hive
pipeline_stages:
-
logfmt:
mapping:
timestamp:
user:
database:
level:
context:
haflog:
static_configs:
-
labels:
__path__: /var/log/postgresql/*log
hive: logs
targets:
- localhost
-
job_name: arch
static_configs:
-
labels:
__path__: /var/log/postgresql/*log.1
hive: archive
targets:
- localhost
server:
grpc_listen_port: 0
http_listen_port: 9080
```
Next step will be creating our dockers that collect, send and visualize sent logs by creating a docker-compose.yaml file in selected directory
```
version: "3.8"
networks:
loki:
services:
loki:
image: grafana/loki:2.6.0
ports:
- "3100:3100"
command: -config.file=/etc/loki/local-config.yaml
networks:
- loki
grafana:
image: grafana/grafana:9.4.3
ports:
- "3000:3000"
networks:
- loki
promtail:
image: grafana/promtail:2.6.0
volumes:
- /var/log/:/var/log/
- /home/dev/docker_volumes/promtail:/etc/promtail
- /var/log/postgresql/:/var/log/postgresql
command: -config.file=/etc/promtail/promtail-config.yml
networks:
- loki
```
In postgresql.conf located in /etc/postgresql/12/main (opened with sudo) change log prefixes to
```
log_line_prefix = 'timestamp="%m" user=%u database=%d level='
```
and log level to highest state
```
log_min_messages = debug5
```
After changes in postgresql.conf you need to restart postgres by ```sudo systemctl restart postgres```
Now we can start our dockers by using this command:
```
docker-compose -f docker-compose.yaml up
```
in localhost:3000 there is located grafana where we need to setup loki as a source of logs and change url to ```http://loki:3100```
Done.Post-1.27.5Michal ZanderMichal Zander