Remote hosts are configured to forward syslog to the collector, where rsyslog writes them under
/var/log/remote. From there the pipeline is:
remote host rsyslog → /var/log/remote/<ip>/*.log
→ log-producer (inotify tail) → RabbitMQ [log_queue]
→ log-consumer (bulk index) → Elasticsearch [<host>-logs-YYYY.MM.DD]
Each host gets its own daily index, named from the source IP — for example 10-128-51-47-logs-2026.08.03.
Documents carry a parsed timestamp, hostname, source_ip, app_name, log_type, message and a derived syslog
severity. Indexing is bulk, and the broker delivery is acknowledged only after the batch is written, so a consumer
restart does not lose the in-flight batch.
← Back to Knowledge Base
/var/log/remote. From there the pipeline is:
remote host rsyslog → /var/log/remote/<ip>/*.log
→ log-producer (inotify tail) → RabbitMQ [log_queue]
→ log-consumer (bulk index) → Elasticsearch [<host>-logs-YYYY.MM.DD]
Each host gets its own daily index, named from the source IP — for example 10-128-51-47-logs-2026.08.03.
Documents carry a parsed timestamp, hostname, source_ip, app_name, log_type, message and a derived syslog
severity. Indexing is bulk, and the broker delivery is acknowledged only after the batch is written, so a consumer
restart does not lose the in-flight batch.