Category: Reliable Penguin

Add Storage To LVM/XFS Partition

May 21, 2014

Let’s assume that you have XFS formatted partition hosted on LVM named “/dev/vg1/gfs1” where “vg1” is the “volume group” and “gfs1” is the “logical volume”. You want to add an additional drive named “dev/xvdc” to the partition to increase storage. Here are the steps: 1. Partition the new drive with fdisk

[root@web root]# fdisk /dev/xvdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x88e6b2eb.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-13054, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-13054, default 13054):
Using default value 13054

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@web root]#

[root@web root]# fdisk /dev/xvdc

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0x88e6b2eb.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

switch off the mode (command 'c') and change display units to

sectors (command 'u').

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 1

First cylinder (1-13054, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-13054, default 13054):

Using default value 13054

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

[root@web root]#

2. Setup /dev/xvdc1 as an LVM physical volume:

[root@web root]# pvcreate /dev/xvdc1
Physical volume "/dev/xvdc1" successfully created

1 2	[root@web root]# pvcreate /dev/xvdc1 Physical volume "/dev/xvdc1" successfully created

3. Add /dev/xvdc1 to the LVM volume group:

[root@web root]# vgextend vg1 /dev/xvdc1
Volume group "vg1" successfully extended

1 2	[root@web root]# vgextend vg1 /dev/xvdc1 Volume group "vg1" successfully extended

You can show volume group detail to see the newly available storage:

[root@web root]# vgdisplay vg1
--- Volume group ---
VG Name vg1
System ID
Format lvm2
Metadata Areas 3
Metadata Sequence No 6
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 3
Act PV 3
VG Size 279.99 GiB
PE Size 4.00 MiB
Total PE 71677
Alloc PE / Size 46078 / 179.99 GiB
Free PE / Size 25599 / 100.00 GiB
VG UUID 7muRk4-U3d4-QJfh-oXFU-REo1-cXI7-z0p1i4

[root@web root]# vgdisplay vg1

--- Volume group ---

VG Name vg1

System ID

Format lvm2

Metadata Areas 3

Metadata Sequence No 6

VG Access read/write

VG Status resizable

MAX LV 0

Cur LV 1

Open LV 1

Max PV 0

Cur PV 3

Act PV 3

VG Size 279.99 GiB

PE Size 4.00 MiB

Total PE 71677

Alloc PE / Size 46078 / 179.99 GiB

Free PE / Size 25599 / 100.00 GiB

VG UUID 7muRk4-U3d4-QJfh-oXFU-REo1-cXI7-z0p1i4

Notice that “Free PE” is 100GB which is the size of our new /dev/xvdc drive. 4. Extend logical volume to use newly available storage:

[root@web root]# lvextend -l +100%FREE /dev/vg1/gfs1
Extending logical volume gfs1 to 279.99 GiB
Logical volume gfs1 successfully resized

[root@web root]# lvextend -l +100%FREE /dev/vg1/gfs1

Extending logical volume gfs1 to 279.99 GiB

Logical volume gfs1 successfully resized

5. Grow the XFS filesystem on “gfs1”:

[root@web root]# xfs_growfs /dev/vg1/gfs1
meta-data=/dev/mapper/vg1-gfs1 isize=256 agcount=10, agsize=5242624 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=47183872, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=10239, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 47183872 to 73397248

[root@web root]# xfs_growfs /dev/vg1/gfs1

meta-data=/dev/mapper/vg1-gfs1 isize=256 agcount=10, agsize=5242624 blks

= sectsz=512 attr=2, projid32bit=0

data = bsize=4096 blocks=47183872, imaxpct=25

= sunit=0 swidth=0 blks

naming =version 2 bsize=4096 ascii-ci=0

log =internal bsize=4096 blocks=10239, version=2

= sectsz=512 sunit=0 blks, lazy-count=1

realtime =none extsz=4096 blocks=0, rtextents=0

data blocks changed from 47183872 to 73397248

And that’s it ..

[root@web root]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/xvda1 41283904 29238956 9947852 75% /
tmpfs 4014008 0 4014008 0% /dev/shm
/dev/mapper/vg1-gfs1 293548036 182781980 110766056 63% /var/www/vhosts

[root@web root]# df

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/xvda1 41283904 29238956 9947852 75% /

tmpfs 4014008 0 4014008 0% /dev/shm

/dev/mapper/vg1-gfs1 293548036 182781980 110766056 63% /var/www/vhosts

Of course there are lots of possible variations on the process. For example maybe you don’t want to allocate all of the free space to one single logical volume

GoAccess Web Log Analyzer

April 11, 2014

Here’s a great webserver log analysis tool. It’s lightweight, fast and runs from the command line on Linux: http://goaccess.prosoftcorp.com/ Just download, build and run with something like:

goaccess -f /var/log/httpd/access_log

1	goaccess -f /var/log/httpd/access_log

I also like generating an HTML report with something like:

cat /var/log/httpd/access_log | goaccess -a > report.html

1	cat /var/log/httpd/access_log \| goaccess -a > report.html

Monitor Web Site Files With Auditd

December 10, 2013

The Linux Auditing System and auditd are a great way to monitor who and when changes are made to the files in your website. To install and configure follow these steps: 1. Install auditd and related utilities:

yum install audit

1	yum install audit

2. Make sure auditd is running:

/sbin/chkconfig --list auditd
auditd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

1 2	/sbin/chkconfig --list auditd auditd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

3. Edit /etc/audit/auditd.conf and change:

action_mail_acct = [your email address]

1	action_mail_acct = [your email address]

This sets any action emails to go to your preferred address. 4. Edit /etc/audit/audit.rules and add a line like this to the bottom:

-w [path_to_website] -p wa -k [key]

1	-w [path_to_website] -p wa -k [key]

So if you website is located at:

/var/www/vhosts/mysite.com/httpdocs

1	/var/www/vhosts/mysite.com/httpdocs

Then a command like:

-w /var/www/vhosts/mysite.com/httpdocs -p wa -k mysite

1	-w /var/www/vhosts/mysite.com/httpdocs -p wa -k mysite

would setup auditing of write and attribute change requests. Events matching this rule would be tagged with the “mysite” key.

/sbin/service auditd restart

1	/sbin/service auditd restart

Audit logs go to:

/var/log/audit/audit.log

1	/var/log/audit/audit.log

Prepend Text To Start Of Every Line

November 12, 2013

Prepend the text “foo” to the start of every line in file. With sed:

sed -i.bak 's/^/foo/' file

1	sed -i.bak 's/^/foo/' file

With perl:

perl -e 'while (<>) {print "foo"}' < infile > outfile

1	perl -e 'while (<>) {print "foo"}' < infile > outfile

Taken from: Prepend ## to every line in a text file

Fix Completion In Shell On Ubuntu

November 6, 2013

Sometimes with a new user on Ubuntu I find that filename auto-completion is missing in the shell. I found two issues that cause this: 1. Ubuntu sets new users to /bin/sh which is a very limited shell. So first step is to change the user to /bin/bash:

chsh -s /bin/bash rpenguin

1	chsh -s /bin/bash rpenguin

2. The /etc/bash.bashrc has filename completion disabled by default. Edit this file and un-comment the appropriate lines. Here’s what it looked like on one server:

# enable bash completion in interactive shells
#if ! shopt -oq posix; then
#  if [ -f /usr/share/bash-completion/bash_completion ]; then
#    . /usr/share/bash-completion/bash_completion
#  elif [ -f /etc/bash_completion ]; then
#    . /etc/bash_completion
#  fi
#fi

# enable bash completion in interactive shells

#if ! shopt -oq posix; then

# if [ -f /usr/share/bash-completion/bash_completion ]; then

# . /usr/share/bash-completion/bash_completion

# elif [ -f /etc/bash_completion ]; then

# . /etc/bash_completion

# fi

#fi

So I changed to:

# enable bash completion in interactive shells
if ! shopt -oq posix; then
  if [ -f /usr/share/bash-completion/bash_completion ]; then
    . /usr/share/bash-completion/bash_completion
  elif [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
  fi
fi

# enable bash completion in interactive shells

if ! shopt -oq posix; then

if [ -f /usr/share/bash-completion/bash_completion ]; then

. /usr/share/bash-completion/bash_completion

elif [ -f /etc/bash_completion ]; then

. /etc/bash_completion

Upon login the new user should now have filename completion features. Also note that by default “vi” is handicapped. Here’s how to upgrade to a full feature set: http://blogs.reliablepenguin.com/2013/10/25/fix-vi-ubuntu