달력

102022  이전 다음

  •  
  •  
  •  
  •  
  •  
  •  
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  •  
  •  
  •  
  •  
  •  

  1. 리파지토리 추가
    $ vi /etc/apt/sources.list.d/dotdeb.org.list
    deb http://packages.dotdeb.org squeeze all
    deb-src http://packages.dotdeb.org squeeze all

  2. 키 등록
    $ wget -q -O - http://www.dotdeb.org/dotdeb.gpg | sudo apt-key add -

  3. 설치
    $ sudo apt-get update
    $ sudo apt-get install redis-server

Posted by zhato

댓글을 달아 주세요

Centralized logging with Graylog2

Design considerations


Centralized logging isn't an easy task, you need to be able to handle very large amounts of data with a lot of write operations and heavy indexing which amounts to ample CPU and memory usage making a scalable text indexing and storage backend extremely important as well as a decoupled architecture. Over the course of a month we evaluated various tools and architectures at Praekelt to find one that worked well (never mind many that just don't work at all).The final configuration uses Ubuntu 12.04 (Precise) for the central server, Graylog2 to receive logs and do analysis etc, RabbitMQ to queue logs for Graylog2 from Logstash which aren't syslog related, then ElasticSearch and MongoDB  which are used by Graylog2 to store logs and stats.

Logging setup

Logstash

Logstash is a good tool, I'm somewhat annoyed by it's version dependencies writing directly to Elasticsearch but no matter, we can use Graylog2 to fill the gap.

Syslog

Ubuntu is nice enough to use rsyslog since quite a while ago. While we do want to make use of remote syslog to collect the usual system logs and dispatch them to the log server, what we absolutely don't want to do is pipe noisy application logs (like HTTP access logs) into syslog. Keeping them all separate has a lot of benefits when it comes to trying to troubleshoot your system later, and avoiding possibly flooding local message facilities away. So while using rsyslog imfile is tempting and easy, it becomes difficult to manage later.

Graylog2

Where Logstash seems to fail on the centralised side of things, Graylog2 is substantially easier to deploy and works on the latest versions of Elasticsearch, and also employs MongoDB as a key store for aggregating statistics which is a very good idea. Graylog2 has some setup guides (and packages) for Ubuntu Lucid - unfortunately Lucid support just ended, so we'll just configure it from scratch in Precise.

Install some necessary packages

root@logger:~# aptitude install build-essential rabbitmq-server openjdk-6-jre-headless mongodb rubygems

Grab the deb packages for Elasticsearch from http://www.elasticsearch.org/download/ and both the Graylog2 server and web interface from http://graylog2.org/download.

root@logger:~# dpkg -i elasticsearch-0.19.3.deb

Take note of any errors or missing dependencies and make sure it starts itself up. You don't need to configure anything else for Elasticsearch. Now get RabbitMQ running a bit more securely.

root@logger:~# rabbitmqctl add_user logging mypassword
Creating user "logging" ...done.
root@logger:~# rabbitmqctl set_permissions logging ".*" ".*" ".*"
Setting permissions for user "logging" in vhost "/" ...done.
root@logger:~# rabbitmqctl delete_user guest
Deleting user "guest" ...done.

First thing to do is setup Logstash to get logs shipped over AMQP.

Client configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
input {
  file {
    type => "syslog"
    path => [ "/var/log/messages", "/var/log/syslog", "/var/log/*.log" ]
  }
  file {
    type => "apache-access"
    path => "/var/log/nginx/access.log"
  }
}
 
output {
  amqp {
    host => "logger.acme.com"
    exchange_type => "fanout"
    name => "rawlogs"
    user => "logging"
    password => "mypassword"
  }
}

Server configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
input {
  amqp {
    type => "all"
    host => "localhost"
    exchange => "rawlogs"
    name => "rawlogs_consumer"
    user => "logging"
    password => "mypassword"
  }
}
 
output {
  stdout { }
  gelf {
    facility => "logstash-gelf"
    host => '127.0.0.1'
  }
}

We leave the stdout output enabled just to check things are working, it's a good idea to disable it when everything is running. We essentially just use Logstash as a broker to get stuff from RabbitMQ into Graylog2 via GELF. Graylog2 does support AMQP directly, but there are some good reasons we do this - namely it doesn't support using AMQP in the same way that Logstash does.

Kickstart both of them the same way, assuming you stored the config as logstash.conf

# java -jar logstash-1.1.0-monolithic.jar agent -f logstash.conf

Next get Graylog2 going. Extract both the server and the web interface into /opt and configure the server. Copy graylog2.conf.example to /etc/graylog2.conf and make the relevant changes. You can use mongodb with or without authentication if it's not accessible externally, we're using authentication here. For setting up MongoDB authentication read more here, which has a bunch of other info on configuring Graylog2 which is worth reading.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
syslog_listen_port = 514
syslog_protocol = udp
 
elasticsearch_url = http://localhost:9200/
elasticsearch_index_name = graylog2
 
force_syslog_rdns = false
 
mongodb_useauth = true
mongodb_user = graylog
mongodb_password = graylog
mongodb_host = localhost
 
#mongodb_replica_set = localhost:27017,localhost:27018,localhost:27019
mongodb_database = graylog2
mongodb_port = 27017
 
mq_batch_size = 4000
mq_poll_freq = 1
 
mq_max_size = 0
 
mongodb_max_connections = 100
mongodb_threads_allowed_to_block_multiplier = 5
 
use_gelf = true
gelf_listen_address = 0.0.0.0
gelf_listen_port = 12201
 
# AMQP
amqp_enabled = false
amqp_subscribed_queues = gqueue:gelf,squeue:syslog
amqp_host = localhost
amqp_port = 5627
amqp_username = guest
amqp_password = guest
amqp_virtualhost = /
 
forwarder_loggly_timeout = 3

You can start Graylog2 up now with 'bin/graylog2ctl start'.

To get the web interface working, do the following

root@logger:/opt/graylog2-web-interface-0.9.6# gem install bundler
root@logger:/opt/graylog2-web-interface-0.9.6# bundle install
root@logger:/opt/graylog2-web-interface-0.9.6# script/rails runserver -e production -p 80

You should now have a running logging system. Now you should go back, get the web interface behind passenger and nginx running as an unprivileged user.


Posted by zhato

댓글을 달아 주세요

apt-get install python-software-properties 


Posted by zhato

댓글을 달아 주세요

 

Installing Graylog2 via Ubuntu Packages

These packages and docs are currently beta. 
The deb's are built on Ubuntu Lucid amd64 however should work on both i386 & amd64.

Please report bugs in this HOWTO or the packaging to me at aussielunix at gmail dot com.

graylog2-server

This installs graylog2-server and it's dependencies (mongodb-stable from 10gen) etc. 
The graylog2-server will install all files to /opt/graylog2-server & a config file at /etc/graylog2..conf. 
Be prepared as the java stuff drags in a lot of deps on a clean minimal Lucid install. (176 packages for me)

1) Install mongodb

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
sudo echo "deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen" | tee -a /etc/apt/sources.list.d/10gen.list
sudo apt-get -y update
sudo apt-get -y install mongodb-10gen

2) install graylog2-server

sudo apt-get install graylog2-server

6) secure mongo - add authentication

  • add an admin user
  • add a user to mongo for collection 'graylog2'
lunix@ubuntu-dev01:~/$ mongo
use admin
db.addUser('admin', 'admin-mongo-passwd')
db.auth('admin', 'admin-mongo-passwd')
use graylog2
db.addUser('grayloguser', 'grayloguser-mongo-passwd')

7) tell graylog2-server about the mongo auth

  • edit /etc/graylog2.conf
mongodb_useauth = true
mongodb_user = grayloguser
mongodb_password = p4ssw0rd

8) turn mongo security on - it's off by default

  • edit /etc/mongodb.conf
auth = true

9) restart mongo

sudo service mongodb restart

10) start graylog2-server

sudo service graylog2-server start

Conclusion

You should now have a working graylog2-server. 
You can check the process tree for a mongodb instance and a java instance and that port UDP/514 is open. 
You can now modify the syslog config on the graylog2-server host to send its data to 127.0.0.1:514 
Move on to graylog2-web install/configure now.

graylog2-web

This installs graylog2-web and some of it's dependencies. 
The graylog2-web package will install all of it's files to /opt/graylog2-web. 
All of the gem dependencies have been vendored in. 
The version of rubygems is too old in the Lucid repositories so I make use of a thirdparty PPA. 
This PPA is from Mackenzie Morgan - a Ubuntu Developer - https://launchpad.net/~maco.m/+archive/ruby

1) add Mackenzie's PPA

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:maco.m/ruby

2) let apt see the new repositories

sudo apt-get update

3) install graylog2-web

sudo apt-get install graylog2-web

4) install budler

sudo gem install bundler

5) review/edit some rails configs:

config/mongoid.yml
confg/email.yml
config/general.yml

6) start graylog2-web as a daemon

script/rails server -eproduction -d -p3000

Posted by zhato

댓글을 달아 주세요

graphite 설치

:: IT/:: Linux 2012. 8. 9. 19:44

 

####################################

# BASIC REQUIREMENTS

# http://graphite.wikidot.com/installation

# http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/

# Last tested & updated 10/13/2011

####################################


sudo apt-get update

sudo apt-get upgrade


wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz

wget http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz

wget http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz

tar -zxvf graphite-web-0.9.9.tar.gz

tar -zxvf carbon-0.9.9.tar.gz

tar -zxvf whisper-0.9.9.tar.gz

mv graphite-web-0.9.9 graphite

mv carbon-0.9.9 carbon

mv whisper-0.9.9 whisper

rm carbon-0.9.9.tar.gz

rm graphite-web-0.9.9.tar.gz

rm whisper-0.9.9.tar.gz

python3.1 libpython3.1 python3.1-minimal

sudo apt-get install --assume-yes apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libapache2-mod-wsgi libaprutil1-ldap memcached python-cairo-dev python-django python-ldap python-memcache python-pysqlite2 sqlite3 erlang-os-mon erlang-snmp rabbitmq-server bzr expect ssh libapache2-mod-python python-setuptools

sudo easy_install django-tagging


####################################

# INSTALL WHISPER

####################################


cd ~/whisper

sudo python setup.py install


####################################

# INSTALL CARBON

####################################


cd ~/carbon

sudo python setup.py install

# CONFIGURE CARBON

####################

cd /opt/graphite/conf

sudo cp carbon.conf.example carbon.conf

sudo cp storage-schemas.conf.example storage-schemas.conf

sudo vi storage-schemas.conf

### edited storage-schemas.conf to be the following

[stats]

priority = 110

pattern = .*

retentions = 10:2160,60:10080,600:262974

###


####################################

# CONFIGURE GRAPHITE (webapp)

####################################


cd ~/graphite

sudo python check-dependencies.py

sudo python setup.py install


# CONFIGURE APACHE

###################

cd ~/graphite/examples

sudo cp example-graphite-vhost.conf /etc/apache2/sites-available/default

sudo cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi

sudo vi /etc/apache2/sites-available/default

# moved 'WSGIImportScript /opt/gr..' to right before virtual host since it gave me an error saying

# WSGIImportScript cannot occur within <VirtualHost> section

# if this path does not exist make it!!!!!!

# /etc/httpd/wsgi

sudo mkdir /etc/httpd

sudo mkdir /etc/httpd/wsgi

sudo /etc/init.d/apache2 reload


####################################

# INITIAL DATABASE CREATION

####################################

cd /opt/graphite/webapp/graphite/

sudo cp local_settings.py.example local_settings.py

sudo python manage.py syncdb

# follow prompts to setup django admin user

sudo chown -R www-data:www-data /opt/graphite/storage/

sudo /etc/init.d/apache2 restart



####################################

# START CARBON

####################################

cd /opt/graphite/

sudo ./bin/carbon-cache.py start


####################################

# SEND DATA TO GRAPHITE

####################################

cd ~/graphite/examples

sudo chmod +x example-client.py

# [optional] edit example-client.py to report data faster

# sudo vi example-client.py

sudo ./example-client.py


Posted by zhato

댓글을 달아 주세요

# ll /etc/localtime
-rw-r--r--  1 root root    2819 Jul 11 18:11 localtime

# cp -pf /usr/share/zoneinfo/Asia/Seoul /etc/localtime

# ll /etc/localtime
-rw-r--r--  1 root root     380 Mar  7 15:12 localtime


# echo "Asia/Seoul" > /etc/timezone

# cat /etc/timezone
Asia/Seoul


Posted by zhato

댓글을 달아 주세요

What is ebtables?

:: IT/:: Linux 2011. 11. 30. 10:42

The ebtables program is a filtering tool for a Linux-based bridging firewall. It enables transparent filtering of network traffic passing through a Linux bridge. The filtering possibilities are limited to link layer filtering and some basic filtering on higher network layers. Advanced logging, MAC DNAT/SNAT and brouter facilities are also included.

 

The ebtables tool can be combined with the other Linux filtering tools (iptables, ip6tables and arptables) to make a bridging firewall that is also capable of filtering these higher network layers. This is enabled through the bridge-netfilter architecture which is a part of the standard Linux kernel.

Posted by zhato

댓글을 달아 주세요