Logstash перестает передавать данные..?

Question

[Wolf KTL]

Поставил Logstash, kibana . еlasticsearch все пятой версии
CentOS Linux release 7.2.1511 (Core)

java -version
java version "1.8.0_73"
Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)

CentOS Linux release 7.2.1511 (Core)

/usr/share/logstash/bin/logstash --configtest
INFO: Logstash requires a setting file which is typically located in $LS_HOME/config or /etc/logstash. If you installed Logstash through a package and are starting it manually please specify the location to this settings file by passing in "--path.settings=/path/.."
ERROR: Failed to load settings file from "path.settings". Aborting... path.setting=/usr/share/logstash/config, exception=Errno::ENOENT, message=>No such file or directory - /usr/share/logstash/config/logstash.yml

[root@kibana ~]# /etc/logstash/conf.d/config.conf -f /etc/logstash/conf.d/config.conf --configtest --debug
-bash: /etc/logstash/conf.d/config.conf: Permission denied

/etc/logstash/conf.d/config.conf

input {
udp {

port => 5514
type => "syslog"
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "mikrotik-%{+YYYY.MM.dd}"
document_type => "%{type}"
}
}

[root@kibana ~]# netstat -ntulp | grep 5514
udp6 213440 0 :::5514 :::* 616/java

logstash-plain.log
[2016-11-10T12:40:05,256][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-10T12:40:05,257][WARN ][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url_info=>{}}
[2016-11-10T12:40:08,012][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-10T12:40:08,013][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>16}
[2016-11-10T12:40:10,259][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}

Данные в кибану идет а потом перестают поступать

Comments

[Wolf KTL]

[2016-11-10T15:29:53,096][INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>"0.0.0.0:5514"}
[2016-11-10T15:29:53,981][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>125}
[2016-11-10T15:29:53,999][INFO ][logstash.pipeline ] Pipeline main started
[2016-11-10T15:29:54,264][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2016-11-10T15:29:59,023][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>["localhost:9200"], :added=>[]}}
[2016-11-10T15:29:59,131][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-10T15:29:59,132][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>2}
[2016-11-10T15:30:01,168][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#

Answer 1

[Wolf KTL]

192.168.100.253:9200
{
"name" : "UhWJ24N",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "p9iMIEe5TDycADQLjehw5A",
"version" : {
"number" : "5.0.0",
"build_hash" : "253032b",
"build_date" : "2016-10-26T04:37:51.531Z",
"build_snapshot" : false,
"lucene_version" : "6.2.0"
},
"tagline" : "You Know, for Search"

Comments

[Макс]

а curl 127.0.0.1:9200 ?

[Wolf KTL]

[root@kibana ~]# curl 127.0.0.1:9200
{
"name" : "UhWJ24N",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "p9iMIEe5TDycADQLjehw5A",
"version" : {
"number" : "5.0.0",
"build_hash" : "253032b",
"build_date" : "2016-10-26T04:37:51.531Z",
"build_snapshot" : false,
"lucene_version" : "6.2.0"
},
"tagline" : "You Know, for Search"
}

[Wolf KTL]

Important

Logstash 5.0 introduces a new way to configure application settings for Logstash through a logstash.yml file.

This file is typically located in ${LS_HOME}/config, or /etc/logstash when installed via packages. Logstash will not be able to start without this file, so please make sure to pass in --path.settings if you are starting Logstash manually after installing it via a package (RPM, DEB).

[Wolf KTL]

и походу в пятой версии нет команды configtest

[Wolf KTL]

заменили на logstash -t

[Wolf KTL]

[root@kibana config]# /usr/share/logstash/bin/logstash -t
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties.
Configuration OK

[Wolf KTL]

Логи : [2016-11-11T12:58:42,333][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-11T12:58:42,334][ERROR][logstash.outputs.elasticsearch] Attempted to send a bulk request to elasticsearch, but no there are no living connections in the connection pool. Perhaps Elasticsearch is unreachable or down? {:error_message=>"No Available connections", :class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::NoConnectionAvailableError", :will_retry_in_seconds=>16}
[2016-11-11T12:58:43,479][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-11T12:58:43,479][WARN ][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url_info=>{}}
[2016-11-11T12:58:48,493][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-11T12:58:48,493][WARN ][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url_info=>{}}
[2016-11-11T12:58:53,517][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}
[2016-11-11T12:58:53,518][WARN ][logstash.outputs.elasticsearch] Elasticsearch output attempted to sniff for new connections but cannot. No living connections are detected. Pool contains the following current URLs {:url_info=>{}}
[2016-11-11T12:58:58,352][WARN ][logstash.outputs.elasticsearch] UNEXPECTED POOL ERROR {:e=>#}

Answer 2

[Goodsmileduck]

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => false
    manage_template => false
    index => "mikrotik-%{+YYYY.MM.dd}"
    document_type => "%{type}"
    }
}

https://discuss.elastic.co/t/logstash-elasticsearc...