|||

The ELK stack consists of, roughly, three parts. The consumer, logstash; the Storage, elasticsearch; and the viewer, kibana.

Elasticsearch Commands

Display cluster details

curl -XGET 'localhost:9200/_cluster/health/?pretty'

Create template to limit shards and replcias

curl -XPUT 'localhost:9200/_template/template_forall?pretty' -H 'Content-Type: application/json' -d'
{
    "template" : "*",
    "order" : 0,
    "settings" : {
        "number_of_shards" : 1,
        "number_of_replicas" : 1
    }
}
'

Display shard details

curl -XGET 'localhost:9200/_cat/shards/logstash-2017.06.06/?human&pretty'

logstash-2017.06.06 0 p STARTED 148712 186.6mb 10.1.10.149 elk1
logstash-2017.06.06 0 r STARTED 148714 186.7mb 10.1.10.144 elk0

Finding Problem Shards ~~~ curl -XGET localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason| grep UNASSIGNED ~~~

Delete an index

curl -XDELETE 'localhost:9200/twitter?pretty'

Troubleshooting Shard allocaiton issues

We’ve ran into some issues with shard allocation once in a while. Here are some good references to help diagnose:

curl -XGET 'localhost:9200/_cat/shards/?human&pretty' | grep UNASSIGNED 
curl -XPUT 'localhost:9200/indexname-2018.06.15/_settings?pretty' -H 'Content-Type: application/json' -d'{"index":{"number_of_replicas" : 0}}' 

Temperature monitor (with or without nagios)

I wrote up a little python script that reads the gpio ports on the raspberry pi for a DS18B20 temperature sensor. It’s probably not perfect, but it seems to work.

 python gpio_check.py --syslogserver 192.168.1.144 --syslogport 1514 --site SlowDownHome --sensor FloorTemp --sensorsystem 28-000006152042 --sensortype temp --measure f -c 65 -w 60 

output looks something like

<SlowDownHome, FloorTemp> Warning: 64.2866 is greater than 60 for SlowDownHome - FloorTemp

OK, the logstash bits now…

input {
  tcp{
    port => 1514
    tags => ["temp"]
  }
  udp{
    port => 1514
    tags => ["temp"]
  }
}
filter {
  if "temp" in [tags] {
    grok {
      match => { "message" => "<%{WORD:site}, %{WORD:sensor}> %{WORD:status}: %{BASE16FLOAT:ftemp:float}"  }
    }
  }
}

Another optional config (if you have multiple types of messages coming in with a differnt format)

input {
  tcp{
    port => 1514
    tags => ["syslog"]
  }
  udp{
    port => 1514
    tags => ["syslog"]
  }
}
filter {
  if "syslog" in [tags] {
    grok {
      match => { "message" => "<%{WORD:site}, %{WORD:sensor}> %{WORD:status}: %{BASE16FLOAT:ftemp:float}"  }
      add_tag => ["temp"]
    }
  }
}

Rolling upgrade (v5.x)

curl -XPUT 'localhost:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'
{
    "transient": {
          "cluster.routing.allocation.enable": "none"
            }
}
'

Sync Flush

curl -XPOST 'localhost:9200/_flush/synced?pretty'

Do upgrade bits

Reenable cluster routing

curl -XPUT 'localhost:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'
{
    "transient": {
          "cluster.routing.allocation.enable": "all"
            }
}
'

Check health of the cluster

curl -XGET 'localhost:9200/_cat/health?pretty'

Elasticsearch Queries

{"query": {"bool":{"must": [{"match":{"field_name": "fieldvalue"}},{"match": {"host":"hostname"}}],"filter":{"range":{ "@timestamp": { "gte": "now-120m", "lte": "now" } }}}},"size":1,"sort":[{"@timestamp":{"order":"desc"}}]}

Nmap logging

Not sure this works any longer… There wasn’t a codec available when last I checked… but that was before now; maybe there is one now?.

install logstash codec

/usr/share/logstash/bin/logstash-plugin install logstash-codec-nmap

config

input {
  http {
    port => 8080
    codec => "nmap"
    tags => ["nmap"]
  }
}
output {
    stdout { codec => rubydebug }  
}

nmap command

nmap 192.168.1.1-254 -oX - | curl -H "x-nmap-target: Test" http://192.168.1.144:8080 --data-binary @-

output looks something like this

{
  "_index": "nmap-2017.05.20",
  "_type": "nmap_host",
  "_id": "AVwmqrxjnyQGj4R3m1i5",
  "_version": 1,
  "_score": null,
  "_source": {
    "addresses": [
      {
        "type": "ipv4",
        "addr": "192.168.1.126"
      },
      {
        "type": "mac",
        "addr": "55:55:55:55:55:55"
      }
    ],
    "traceroute": null,
    "type": "nmap_host",
    "ports": [
      {
        "number": 8008,
        "reason": "syn-ack",
        "protocol": "tcp",
        "service": {
          "protocol": null,
          "product": null,
          "hostname": null,
          "fingerprint_method": "table",
          "confidence": 3,
          "name": "http",
          "fingerprint": null,
          "device_type": null,
          "ssl": false
        },
        "state": "open"
      },
      {
        "number": 8009,
        "reason": "syn-ack",
        "protocol": "tcp",
        "service": {
          "protocol": null,
          "product": null,
          "hostname": null,
          "fingerprint_method": "table",
          "confidence": 3,
          "name": "ajp13",
          "fingerprint": null,
          "device_type": null,
          "ssl": false
        },
        "state": "open"
      },
      {
        "number": 9000,
        "reason": "syn-ack",
        "protocol": "tcp",
        "service": {
          "protocol": null,
          "product": null,
          "hostname": null,
          "fingerprint_method": "table",
          "confidence": 3,
          "name": "cslistener",
          "fingerprint": null,
          "device_type": null,
          "ssl": false
        },
        "state": "open"
      }
    ],
    "mac": "55:55:55:55:55:55",
    "times": {
      "3781": 0,
      "rttvar": 0
    },
    "hostname": null,
    "ipv4": "192.168.1.126",
    "ipv6": null,
    "@version": "1",
    "host": "192.168.1.113",
    "id": "560d18f4-9127-4f71-a049-15158410f07f-h6",
    "headers": {
      "http_accept": "*/*",
      "content_type": "application/x-www-form-urlencoded",
      "http_x_nmap_target": "Test",
      "request_path": "/",
      "http_version": "HTTP/1.1",
      "request_method": "POST",
      "http_host": "192.168.1.144:8080",
      "request_uri": "/",
      "content_length": "12828",
      "http_expect": "100-continue",
      "http_user_agent": "curl/7.54.0"
    },
    "address": "192.168.1.126",
    "os": null,
    "ip": "192.168.1.126",
    "end_time": "2017-05-20T16:22:13.000Z",
    "version": "7.40",
    "uptime": null,
    "tags": [
      "nmap"
    ],
    "start_time": "2017-05-20T16:22:10.000Z",
    "@timestamp": "2017-05-20T16:22:22.440Z",
    "arguments": "nmap -oX - 192.168.1.1-254",
    "scan_id": "560d18f4-9127-4f71-a049-15158410f07f",
    "status": {
      "reason": "arp-response",
      "state": "up"
    }
  },
  "fields": {
    "end_time": [
      1495297333000
    ],
    "start_time": [
      1495297330000
    ],
    "@timestamp": [
      1495297342440
    ]
  },
  "sort": [
    1495297342440
  ]
}
Up next Book Notes No order to this list… Other than Woodcraft and Camping; If you have any interest in the outdoors; Buy a couple of these; one to keep and one to Recommended Tech Password Protection Web Browsers Laptops Desktops Antivirus VPN Entertainment Linux vim Use a password manager. Lastpass; F-secure; Abine Blur;
Latest posts System Design Cards About Music, Finding the good stuff. “Richie Pan’s America” Font shirt designs Now Learning Graphical Design Python Notes Deploy to a Remote Docker registry DMented 'zine for DM's and Character Sheets for Players Online Security Find Notes Kawasaki Vulcan S 650 Notes Honda Rebel 300 Notes Samba Notes Things worth noting My Software Friends Favorites Recommended Tech Elasticsearch Notes Book Notes APRS Notes vim notes Nmap Notes 4runner Notes Bash Notes Raspberry PI Zero quickstart Notes on Van Life Ubuntu 16.04 or 18.04 Installing Dropbox on Arch Linux Arch Linux Notes