Design considerations
Centralized logging isn't an easy task, you need to be able to handle very large amounts of data with a lot of write operations and heavy indexing which amounts to ample CPU and memory usage making a scalable text indexing and storage backend extremely important as well as a decoupled architecture. Over the course of a month we evaluated various tools and architectures at Praekelt to find one that worked well (never mind many that just don't work at all).The final configuration uses Ubuntu 12.04 (Precise) for the central server, Graylog2 to receive logs and do analysis etc, RabbitMQ to queue logs for Graylog2 from Logstash which aren't syslog related, then ElasticSearch and MongoDB which are used by Graylog2 to store logs and stats.
Logstash is a good tool, I'm somewhat annoyed by it's version dependencies writing directly to Elasticsearch but no matter, we can use Graylog2 to fill the gap.
Ubuntu is nice enough to use rsyslog since quite a while ago. While we do want to make use of remote syslog to collect the usual system logs and dispatch them to the log server, what we absolutely don't want to do is pipe noisy application logs (like HTTP access logs) into syslog. Keeping them all separate has a lot of benefits when it comes to trying to troubleshoot your system later, and avoiding possibly flooding local message facilities away. So while using rsyslog imfile is tempting and easy, it becomes difficult to manage later.
Where Logstash seems to fail on the centralised side of things, Graylog2 is substantially easier to deploy and works on the latest versions of Elasticsearch, and also employs MongoDB as a key store for aggregating statistics which is a very good idea. Graylog2 has some setup guides (and packages) for Ubuntu Lucid - unfortunately Lucid support just ended, so we'll just configure it from scratch in Precise.
Install some necessary packages
root@logger:~# aptitude install build-essential rabbitmq-server openjdk-6-jre-headless mongodb rubygems
Grab the deb packages for Elasticsearch from http://www.elasticsearch.org/download/ and both the Graylog2 server and web interface from http://graylog2.org/download.
root@logger:~# dpkg -i elasticsearch-0.19.3.deb
Take note of any errors or missing dependencies and make sure it starts itself up. You don't need to configure anything else for Elasticsearch. Now get RabbitMQ running a bit more securely.
root@logger:~# rabbitmqctl add_user logging mypassword
Creating user "logging" ...done.
root@logger:~# rabbitmqctl set_permissions logging ".*" ".*" ".*"
Setting permissions for user "logging" in vhost "/" ...done.
root@logger:~# rabbitmqctl delete_user guest
Deleting user "guest" ...done.
First thing to do is setup Logstash to get logs shipped over AMQP.
Client configuration
Server configuration
We leave the stdout output enabled just to check things are working, it's a good idea to disable it when everything is running. We essentially just use Logstash as a broker to get stuff from RabbitMQ into Graylog2 via GELF. Graylog2 does support AMQP directly, but there are some good reasons we do this - namely it doesn't support using AMQP in the same way that Logstash does.
Kickstart both of them the same way, assuming you stored the config as logstash.conf
# java -jar logstash-1.1.0-monolithic.jar agent -f logstash.conf
Next get Graylog2 going. Extract both the server and the web interface into /opt and configure the server. Copy graylog2.conf.example to /etc/graylog2.conf and make the relevant changes. You can use mongodb with or without authentication if it's not accessible externally, we're using authentication here. For setting up MongoDB authentication read more here, which has a bunch of other info on configuring Graylog2 which is worth reading.
You can start Graylog2 up now with 'bin/graylog2ctl start'.
To get the web interface working, do the following
root@logger:/opt/graylog2-web-interface-0.9.6# gem install bundler
root@logger:/opt/graylog2-web-interface-0.9.6# bundle install
root@logger:/opt/graylog2-web-interface-0.9.6# script/rails runserver -e production -p 80
You should now have a running logging system. Now you should go back, get the web interface behind passenger and nginx running as an unprivileged user.
These packages and docs are currently beta.
The deb's are built on Ubuntu Lucid amd64 however should work on both i386 & amd64.
Please report bugs in this HOWTO or the packaging to me at aussielunix at gmail dot com.
This installs graylog2-server and it's dependencies (mongodb-stable from 10gen) etc.
The graylog2-server will install all files to /opt/graylog2-server & a config file at /etc/graylog2..conf.
Be prepared as the java stuff drags in a lot of deps on a clean minimal Lucid install. (176 packages for me)
1) Install mongodb
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
sudo echo "deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen" | tee -a /etc/apt/sources.list.d/10gen.list
sudo apt-get -y update
sudo apt-get -y install mongodb-10gen
2) install graylog2-server
sudo apt-get install graylog2-server
6) secure mongo - add authentication
lunix@ubuntu-dev01:~/$ mongo use admin db.addUser('admin', 'admin-mongo-passwd') db.auth('admin', 'admin-mongo-passwd') use graylog2 db.addUser('grayloguser', 'grayloguser-mongo-passwd')
7) tell graylog2-server about the mongo auth
mongodb_useauth = true mongodb_user = grayloguser mongodb_password = p4ssw0rd
8) turn mongo security on - it's off by default
auth = true
9) restart mongo
sudo service mongodb restart
10) start graylog2-server
sudo service graylog2-server start
You should now have a working graylog2-server.
You can check the process tree for a mongodb instance and a java instance and that port UDP/514 is open.
You can now modify the syslog config on the graylog2-server host to send its data to 127.0.0.1:514
Move on to graylog2-web install/configure now.
This installs graylog2-web and some of it's dependencies.
The graylog2-web package will install all of it's files to /opt/graylog2-web.
All of the gem dependencies have been vendored in.
The version of rubygems is too old in the Lucid repositories so I make use of a thirdparty PPA.
This PPA is from Mackenzie Morgan - a Ubuntu Developer - https://launchpad.net/~maco.m/+archive/ruby
1) add Mackenzie's PPA
sudo apt-get install python-software-properties sudo add-apt-repository ppa:maco.m/ruby
2) let apt see the new repositories
sudo apt-get update
3) install graylog2-web
sudo apt-get install graylog2-web
4) install budler
sudo gem install bundler
5) review/edit some rails configs:
config/mongoid.yml confg/email.yml config/general.yml
6) start graylog2-web as a daemon
script/rails server -eproduction -d -p3000
####################################
# BASIC REQUIREMENTS
# http://graphite.wikidot.com/installation
# http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/
# Last tested & updated 10/13/2011
####################################
sudo apt-get update
sudo apt-get upgrade
wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz
wget http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz
tar -zxvf graphite-web-0.9.9.tar.gz
tar -zxvf carbon-0.9.9.tar.gz
tar -zxvf whisper-0.9.9.tar.gz
mv graphite-web-0.9.9 graphite
mv carbon-0.9.9 carbon
mv whisper-0.9.9 whisper
rm carbon-0.9.9.tar.gz
rm graphite-web-0.9.9.tar.gz
rm whisper-0.9.9.tar.gz
python3.1 libpython3.1 python3.1-minimal
sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libapache2-mod-wsgi libaprutil1-ldap memcached python-cairo-dev python-django python-ldap python-memcache python-pysqlite2 sqlite3 erlang-os-mon erlang-snmp rabbitmq-server bzr expect ssh libapache2-mod-python python-setuptools
sudo easy_install django-tagging
####################################
# INSTALL WHISPER
####################################
cd ~/whisper
sudo python setup.py install
####################################
# INSTALL CARBON
####################################
cd ~/carbon
sudo python setup.py install
# CONFIGURE CARBON
####################
cd /opt/graphite/conf
sudo cp carbon.conf.example carbon.conf
sudo cp storage-schemas.conf.example storage-schemas.conf
sudo vi storage-schemas.conf
### edited storage-schemas.conf to be the following
[stats]
priority = 110
pattern = .*
retentions = 10:2160,60:10080,600:262974
###
####################################
# CONFIGURE GRAPHITE (webapp)
####################################
cd ~/graphite
sudo python check-dependencies.py
sudo python setup.py install
# CONFIGURE APACHE
###################
cd ~/graphite/examples
sudo cp example-graphite-vhost.conf /etc/apache2/sites-available/default
sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
sudo vi /etc/apache2/sites-available/default
# moved 'WSGIImportScript /opt/gr..' to right before virtual host since it gave me an error saying
# WSGIImportScript cannot occur within <VirtualHost> section
# if this path does not exist make it!!!!!!
# /etc/httpd/wsgi
sudo mkdir /etc/httpd
sudo mkdir /etc/httpd/wsgi
sudo /etc/init.d/apache2 reload
####################################
# INITIAL DATABASE CREATION
####################################
cd /opt/graphite/webapp/graphite/
sudo cp local_settings.py.example local_settings.py
sudo python manage.py syncdb
# follow prompts to setup django admin user
sudo chown -R www-data:www-data /opt/graphite/storage/
sudo /etc/init.d/apache2 restart
####################################
# START CARBON
####################################
cd /opt/graphite/
sudo ./bin/carbon-cache.py start
####################################
# SEND DATA TO GRAPHITE
####################################
cd ~/graphite/examples
sudo chmod +x example-client.py
# [optional] edit example-client.py to report data faster
# sudo vi example-client.py
sudo ./example-client.py
# ll /etc/localtime
-rw-r--r-- 1 root root 2819 Jul 11 18:11 localtime
# cp -pf /usr/share/zoneinfo/Asia/Seoul /etc/localtime
# ll /etc/localtime# echo "Asia/Seoul" > /etc/timezone
# cat /etc/timezone
Asia/Seoul
The ebtables program is a filtering tool for a Linux-based bridging firewall. It enables transparent filtering of network traffic passing through a Linux bridge. The filtering possibilities are limited to link layer filtering and some basic filtering on higher network layers. Advanced logging, MAC DNAT/SNAT and brouter facilities are also included.
The ebtables tool can be combined with the other Linux filtering tools (iptables, ip6tables and arptables) to make a bridging firewall that is also capable of filtering these higher network layers. This is enabled through the bridge-netfilter architecture which is a part of the standard Linux kernel.
댓글을 달아 주세요