Feb

2010

Archive for the ‘Linux / Unix’ Category

by @ 20:20

Permalink

HOW TO: Perform system backup in Ubuntu – Part 2 – Restore

Sunday, February 7th, 2010

The restore will need to be perfomred as root

tar xvpzf filename.tgz -C /

the mkdir options are mandatory, and you will notice that they are the directories that are excluded from the backup. this is fine, but the folders do need to be replaced BEFORE YOU DO ANYTHING AFTER THE RESTORE

mkdir proc
mkdir lost+found
mkdir mnt
mkdir sys
mkdir media
mkdir cdrom
mkdir home

now reboot

everything should now be cushdy

Page 2 of 4«123 4 »

HOW TO: Perform system backup in Ubuntu – Part 1 – Backup

Sunday, February 7th, 2010

I had a few issues with my Ubuntu system after installing KDE4. I don’t think KDE4 caused the issue, but a combination of the software I use and the things I do was causing me login issues and X session headaches. I would have programs close immediately after opening, which I could only solve with a reboot. Files were locked, preventing me from running applications like the Terminal and a few other apps. System updates would fail for an unknown reason.

I had to rebuild the system. I saved off all my personal data and reinstalled everything.

Once I was back at my desktop, after configuring my dual screen setup on an Nvidia card and getting my email and browser back the way I like them I decided it was time to make some changes.

The /home folder needed to be moved from the OS drive, (that’s the next post) and I needed a way to back everything up, and I mean everything. Because of the way Linux structures its OS it is geniusly easy to backup the system. You cannot do this on windows without expensive software to ghost the drive.

What we are going to do here is use TAR to compress and zip up the entire root structure, with a few exclusions.

NOTE
I run the command as sudo from my own account. I am pretty sure it would be benneficial to quit all X sessions and drop to the command prompt rather than run from a live session. I have no evidence to suggest that doing this from the desktop will cause any problems. If you have a suggestion on this please comment below.

I will show you how I backed up my root directory and then I will supply a generic command that will suffice for most people. My command is slightly different as I moved my /home folder onto a seperate hard drive. I will cover my reasoning for this in my next post.

This is the command I use to backup my OS

sudo tar cvpzf /media/500/000_Backups/System/system_backup-$(date +%d-%m-%Y).tgz –exclude=/proc –exclude=lost+found –exclude=/mnt –exclude=/media –exclude=/cdrom –exclude=/home –exclude=/old_home /

As you can see I have excluded the /home directory in my backups as it is on a seperate partition. Now, lets break this down into its component parts so you can better understand how this works.

“tar” is obvious. this is the application we will be using to create the archived backup. Make sure you have tar installed. It should be by default but just incase you will need to run this command in the Terminal

sudo apt-get install tar

or, click <a href=”apt:tar”>here</a>

the tar parameters “cvpzf” are as follows.
c – this tells tar that we are creating an archive
v – verbose mode, this command outputs each file being archived into the terminal window
p – retain all file permissions as is
z – compress as gzip – use j for bzip2 but make sure you change the file extension to tar.bz2
f – filename follows this parameter.

system_backup-$(date +%d-%m-%Y).tgz – this is the filename followed by the date of the day the backup is made, in the format dd/mm/yyyy

I chose to date it as I will have incremental backups running via cron. This is a little out of the scope of this post, but happy to help those if you post a comment and ask.

–exclude=/ – we are going to exclude some folders that are not required in the backup. proc, lost+found and cdrom are not required in the backup, and if you do not want to backup any drives, dvds or cdroms that are mounted inside mnt or media, make sure you exclude those as well or the backup could become extremely large.

/ – this is the folder we are backing up. and as we are backing up the whole system we need to backup from the root directory which is /

Now we have our backup command explained you are ready to get down to running your own. Obvioulsy please dont use the one above as it is for my machine, below is the generic code that should suffice 99% of people.

NOTE
please change the “pathtofolder” to a location you would like to save the file once it is completed.
this will take a while once you execute the command, so don’t be tempted to cancel the operation.

sudo tar cvpzf /pathtofolder/system_backup-$(date +%d-%m-%Y).tgz –exclude=/proc –exclude=lost+found –exclude=/mnt –exclude=/media –exclude=/cdrom /

Now that you have your backup, you are going to need to restore it someday. Below are simple instructions on how to extract and restore your data

sudo tar xvpzf system_backup-dd-mm-yyy.tgz -C /

the mkdir options below are mandatory, and you will notice that they are the directories that are excluded from the backup. this is fine, but the folders do need to be replaced BEFORE YOU DO ANYTHING AFTER THE RESTORE

mkdir proc
mkdir lost+found
mkdir mnt
mkdir media
mkdir cdrom

Now reboot your PC and your data will be restored. This includes all applications, mail and broswer settings, everything, except the folders you did not backup.

Page 2 of 4«123 4 »

How to: Install Ubuntu packages with HTML link

Saturday, March 15th, 2008

Apturl allows you to install Ubuntu packages from a website using the following syntax: apt:packagename (where packagename is the name of the package you wish to install

For example, I wanted to install a new pixel ruler (link courtesy of Borntwisted) called KRuler. If I was to install this directly from my web browser I would need to write the link in my HTML like so:

<a href=”apt:kruler”>Click here to install Kruler</a>

This would then give me the following link:

Click here to install Kruler

If you click the link (providing you are using Ubuntu as your Operating System) you will get the follwoing:

Once you click ok, and type in your admin password you will get the download screen as it downloads the package ready for installation

And then the obligatory install screen will be shown:

This is a perfect tool for bloggers, how to’ers and manual writers writing for Ubuntu installs etc. I know I will be using it in all my future blogs and how to’s in the future.

Go ahead, give it a shot

Page 2 of 4«123 4 »

How to: Increase ext3 and reiserfs performance

Tuesday, March 11th, 2008

NOTE: This How To: is for Ubuntu Linux (all versions) but should work on any version of Linux that supports the ext3 filesystem and reiserfs filesystem. This How To: is offered as is and I cannot take responsibility for any problems you suffer as a result of modifying yoru core system files.

ext3 Filesystem

The ext3 or third extended filesystem is a journalled file system that is commonly used by the Linux operating system. It is the default file system for many popular Linux distributions

Features of ext3 File System

The ext3 file system is essentially an enhanced version of the ext2 file system. These improvements provide the following advantages

Availability

After an unexpected power failure or system crash, each mounted ext2 file system on the machine must be checked for consistency by the e2fsck program. This is a time-consuming process that can delay system boot time significantly, especially with large volumes containing a large number of files. During this time, any data on the volumes is unreachable.

The journaling provided by the ext3 file system means that this sort of file system check is no longer necessary after an unclean system shutdown. The only time a consistency check occurs using ext3 is in certain rare hardware failure cases, such as hard drive failures. The time to recover an ext3 file system after an unclean system shutdown does not depend on the size of the file system or the number of files; rather, it depends on the size of the journal used to maintain consistency. The default journal size takes about a second to recover, depending on the speed of the hardware.

Data Integrity

The ext3 file system provides stronger data integrity in the event that an unclean system shutdown occurs. The ext3 file system allows you to choose the type and level of protection that your data receives. By default, Most Linux Distributions configures ext3 volumes to keep a high level of data consistency with regard to the state of the file system.

Speed

Despite writing some data more than once, ext3 has a higher throughput in most cases than ext2 because ext3’s journaling optimizes hard drive head motion. You can choose from three journaling modes to optimize speed, but doing so means trade offs in regards to data integrity.

Easy Transition

It is easy to change from ext2 to ext3 and gain the benefits of a robust journaling file system without reformatting.

ReiserFS

ReiserFS is a general-purpose, journaled computer file system designed and implemented by a team at Namesys led by Hans Reiser.ReiserFS is currently supported on Linux and may be included in other operating systems in the future. Introduced in version 2.4.1 of the Linux kernel, it was the first journaling file system to be included in the standard kernel.

ReiserFS Features

ReiserFS has fast journaling, which means that you don’t spend your life waiting for fsck every time your laptop battery dies, or the UPS for your mission critical server gets its batteries disconnected accidentally by the UPS company’s service crew, or your kernel was not as ready for prime time as you hoped, or the silly thing decides you mounted it too many times today.

ReiserFS is based on fast balanced trees. Balanced trees are more robust in their performance, and are a more sophisticated algorithmic foundation for a file system. When we started our project, there was a consensus in the industry that balanced trees were too slow for file system usage patterns. We proved that if you just do them right they are better–take a look at the benchmarks. We have fewer worst case performance scenarios than other file systems and generally better overall performance. If you put 100,000 files in one directory, we think its fine; many other file systems try to tell you that you are wrong to want to do it.

ReiserFS is more space efficient. If you write 100 byte files, we pack many of them into one block. Other file systems put each of them into their own block. We don’t have fixed space allocation for inodes. That saves 6% of your disk.

Ext3 & ReiserFS has three kinds of journaling methods

1) Journal Data Writeback
2) Journal Data Ordered
3) Journal Data

By default the the 2nd method is used.To speed things up we will make it use method 1. The price to pay is that it may allow old data to appear in files after a crash and journal recovery.

How to make ext3 or reiserfs use journal data writeback

First you need to take a back up of the fstab file using the following command

sudo cp /etc/fstab /etc/fstab.orig

Edit the /etc/fstab file using the following command

sudo gedit /etc/fstab

Add the thing marked in bold to your fstab root mount line.

/dev/hda1 / ext3 defaults,errors=remount-ro,atime,auto,rw,dev,exec,suid,nouser,data=writeback 0 1

Save that file and exit

You need to take a Grubmenu file backup using the following command

sudo cp /boot/grub/menu.lst /boot/grub/menu.lst.orig

Now you need to edit the grub menu list file using the following command

sudo vi /boot/grub/menu.lst

look for the following two lines

# defoptions=quiet splash
# altoptions=(recovery mode) single

change to

# defoptions=quiet splash rootflags=data=writeback
# altoptions=(recovery mode) single rootflags=data=writeback

Save that file and exit

Now you need to update the grub using the following command

sudo update-grub

the added flags will automatically be added to the kernel line and stay there in case of kernel update

Changes to Ext3 FileSystem Only

Note:- tune2fs only works for ext3. Reiserfs can’t change the journal method

Before rebooting change the filesystem manually to writeback using the following command

sudo tune2fs -o journal_data_writeback /dev/hda1

Check that it is running or not using the following command

sudo tune2fs -l /dev/hda1

Remove update of access time for files

Having the modified time change you can understand but having the system updating the access time every time a file is accessed is not to my liking. According to the manual the only thing that might happen if you turn this off is that when compiling certain things the make might need that info.

To change this do the following

sudo vi /etc/fstab

add the following marked in bold

/dev/hda1 / ext3 defaults,errors=remount-ro,noatime,auto,rw,dev,exec,suid,nouser,data=writeback 0 1

Now reboot and enjoy a much faster system

Page 2 of 4«123 4 »

Mount network shares at boot. (Ubuntu)

Monday, March 10th, 2008

If, like me you have your media and web server kept on another machine, and get annoyed having to put your username and password in each time you want to grab a file or listen to music, then this is for you.

This is the most secure method I could come up with, simply put, if your main machine is not logged on as you, then the shares cannot be accessed without the username and password. When you are logged on these details will be called from a file that is owned by the root user.

This tutorial will assume you have samba already installed on your media/web server. Without it, this will not work.

The following must be done on the SERVER machine

sudo gedit /etc/samba/smb.conf

I am using GEDIT here to make it easier for you to amend the file. I much prefer PICO, or you may wish to use VI, if your balls are bigger than King Kongs.

Make the following changes:

workgroup = “WORKGROUP”

Note: The name of the workgroup (in quotes) must be the same on your windows and/or your Ubuntu box. If you do not know the name or your workgroup, look under “My Network Places” on your windows box, select “Microsoft Windows Network” and you will see a list of your workgroups. Default workgroup on windows XP, home edition is MSHOME (not WORKGROUP).

underneath it, add

netbios name = name_of_your_server (no spaces)

For example

netbios name = my_new_smb_server

Make sure the “security” option is set to “user” before moving on.

Scroll down to the bottom of the file and add the following lines:

[Name_of_folder]
path = /path/to/file
browseable = yes
writable = yes
read only = no
guest ok = no
create mask = 0644
directory mask = 0755
force user = root
force group = root

Do not forget to change the name of the folder and the path to the folder. The name you give this folder will appear in the network folder on your Ubuntu box. ie. Music or Vids etc. and the PATH will be something like this path = /home/user/Music. You will get the jist of it.

Save the changes and close the file.

Restart the samba daemon

sudo /etc/init.d/samba restart

The following must be done on YOUR computer

Open a terminal session and type the following commands one by one, allowing each of them to finish before executing the next one.

sudo apt-get install samba
sudo apt-get install smbfs

Now that you have the core files installed we need to do some configuration.

All folders that you wish to have mounted go into a root level folder called /media. This is much nicer than previous and other version of linux where it was called /mnt. now most will know this means MOUNT but some may not. Anyway, lets get on with it.

Lets make that new directory. If you decided to call your share name “Music” in the part above, you would do this:

sudo mkdir /media/Music

Obviously, you name it how you want it. for example, I have many shares such as /media/Vids, /media/Music, /media/WWW etc etc.

Now type the following command

gksudo gedit /root/.smbcredentials

This will create a hidden file called .smbcredentials in your root folder. We will store your username and password in this file for ease of access later on

Insert the following lines into the file: (substituting “yourusername” and “yourpassword” for your ubuntu machine login details)

username=yourusername
password=yourpassword

Save and exit the file and issue these commands one by one:

sudo chmod 700 /root/.smbcredentials
sudo cp /etc/fstab /etc/fstab_backup
gksudo gedit /etc/fstab

Now you should be presented with another file. In here we will make the mods that mount those folders on boot up so you don’t have to touch them again. Put the following line at the bottom of the file, making the changes to the location of your server machine. for example, mine is at 192.168.1.125 so I would do this