Linux on the bin

Linux: Replace Disk in MD

Mon, 28 Dec 2015 00:00:00 +0000

On my lab machines I use a pair of physical disks, configured in a RAID 1 with MD, with LVM on top of that. This gives me a lot of flexibility with full redundancy. However recently I had a failure of a drive.

Identify the Failure

Here is what MD was telling me after I identified and removed the drive from the chassis to send it off for an RMA, notice how each device only has one member (md0 has sda1, md1 has sda2), this does not make a good RAID 1. But this is easily resolved.

Bash: Dynamic Volume Group Mounter

Wed, 16 Sep 2015 00:00:00 +0000

This is similar, but an enhanced version of my custom_mount.sh previously published here: Bash: Automatically Mount File Systems on Volume Group if Present. This previous version was just a simple init script. This updated version is designed for newer Linux distributions that use Systemd instead of Init.

The Code

The majority of the work is done by dvgmounter.sh. It will check if the Volume Group is present, if it is it will parse the /etc/fstab file looking for entries that are marked noauto, then it will test if that file system is already mounted, if it is not it will mount it.

Fedora 20: Use PPTP VPN

Tue, 15 Sep 2015 00:00:00 +0000

I have had this one in the queue for a while (I don’t even run of Fedora 20 anymore). This one will be short and sweet, because there really isn’t a bunch of discussion that needs to go into it.

By default Fedora 20 will block outbound PPTP VPN connections.

 bash
 
 
firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -p gre -j ACCEPT
firewall-cmd --direct --add-rule ipv6 filter INPUT 0 -p gre -j ACCEPT

Reload the rules.

Fedora 20: Firefox Reports Flash as Vulnerable

Wed, 04 Mar 2015 00:00:00 +0000

This problem starts with Firefox reporting that your flash-plugin is out of date. This report looks like this and disables all flash.

After this we will take a look Mozilla’s Plugin Check to see what it thinks is going on.

Now here we can see that version 11.2.202.440 is vulnerable. We will then check about:plugins to see if it agrees.

Again this is also reporting 11.2.202.440, so there must be a problem, but it also tells us that there is an update available. Now I run regular yum updates on this machine, and I actually noticed flash-plugin was updated just a few hours prior to seeing this alert. So lets check the installed version.

Linux KVM: Bridging a Bond on CentOS 6.5

Mon, 27 Oct 2014 00:00:00 +0000

Today we are going to hop back into the KVM fray, and take a look at using CentOS as a hypervisor., and configuring very resilient network connections to support our guests. Of course these instructions should be valid on Red Hat Linux and Oracle Linux as well, though there is a little more to be done around getting access to the repos on those distributions…

Enable Bonding

I am assuming this is a first build for you, so this step might not be applicable, but it won’t hurt anything.

Install MySQL Server Community Edition on Fedora 20

Tue, 01 Jul 2014 00:00:00 +0000

Today we are going to go through the process of installing MySQL Server Community Edition on Fedora 20. You could of course simply use MariaDB, but this will focus on the Community Edition.

Install the MySQL Repository

We will need to install the MySQL Community Repository, the MySQL project hosts an RPM file with the repository. First lets look at the repositories that I have already.

 bash
 
 
yum repolist
Loaded plugins: langpacks, refresh-packagekit
repo id repo name status
adobe-flashplayer Adobe Flash Player - x86_64 2
fedora/20/x86_64 Fedora 20 - x86_64 38,597
google-chrome Google Chrome - x86_64 3
oracle-virtualbox/20/x86_64 Oracle VirtualBox - x86_64 10
rpmfusion-free/20/x86_64 RPM Fusion for Fedora 20 - Free 468
rpmfusion-free-updates/20/x86_64 RPM Fusion for Fedora 20 - Free - Updates 464
rpmfusion-nonfree/20/x86_64 RPM Fusion for Fedora 20 - Nonfree 203
rpmfusion-nonfree-updates/20/x86_64 RPM Fusion for Fedora 20 - Nonfree - Updates 299
updates/20/x86_64 Fedora 20 - x86_64 - Updates 17,616
repolist: 57,662

Now lets install the release file. Keep in mind, you can do a separate wget or curl to download the file, or you can use the command as I have. As of the time of this writing there are also repositories for Fedora 18 and 19 and Enterprise Linux 5, 6 and 7 (beta). Though the only procedures I have validated are the Fedora 20, they should all work the same.

SQL Developer Crash on Fedora 20

Tue, 01 Jul 2014 00:00:00 +0000

I ran into a painful issue on Fedora 20 with SQL Developer. Basically every time it was launched via the shortcut it would go through loading, and then disappear.

Manual Invocation of SQL Developer

When launching it via the script itself it gives us a little more information.

 bash
 
 
/opt/sqldeveloper/sqldeveloper.sh

Oracle SQL Developer
Copyright (c) 1997, 2013, Oracle and/or its affiliates. All rights reserved.

 

LOAD TIME : 279#
A fatal error has been detected by the Java Runtime Environment:
#
SIGSEGV (0xb) at pc=0x00000038a1e64910, pid=12726, tid=140449865832192
#
JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 1.7.0_40-b43)
Java VM: Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode linux-amd64 compressed oops)
Problematic frame:
C 0x00000038a1e64910
#
Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
An error report file with more information is saved as:
/opt/sqldeveloper/sqldeveloper/bin/hs_err_pid12726.log
[thread 140449881597696 also had an error]
#
If you would like to submit a bug report, please visit:
http://bugreport.sun.com/bugreport/crash.jsp
#
/opt/sqldeveloper/sqldeveloper/bin/../../ide/bin/launcher.sh: line 611: 12726 Aborted (core dumped) ${JAVA} "${APP_VM_OPTS[@]}" ${APP_ENV_VARS} -classpath ${APP_CLASSPATH} ${APP_MAIN_CLASS} "${APP_APP_OPTS[@]}"

I also noticed, that while executing as root it worked. However that clearly isn’t the “solution”

Voyage Linux: Dialog Error with Apt

Tue, 01 Jul 2014 00:00:00 +0000

This can happen on other Linux distributions, however in this case I found it on Voyage Linux, which is a Linux distribution for embedded hardware.

The Error

Here we are dealing with an annoyance whenever you use apt-get or aptitude.

 text
 
 
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline

The Fix

Simply install dialog, which is the package it is not finding. This will no longer need the failback to readline.

Voyage Linux: Locale Error with Apt

Tue, 01 Jul 2014 00:00:00 +0000

Voyage Linux is an embedded linux distribution. I use it on some ALIX boards I have lying around, it is very stripped down, and as such there are a few annoyances which we have to fix.

The Error

This issue happens when attempting to install/upgrade packages using apt-get or aptitude.

 text
 
 
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LANG = "en_US.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

The Fix

Oracle Linux 6: Create an OCFS2 Cluster and Filesystem

Mon, 04 Mar 2013 00:00:00 +0000

Today we are going to go through the process of creating a clustered file system on a pair of Oracle Linux 6.3 nodes. This exercise is not very resource intensive. I am using two VMs each with 1GB of RAM a single CPU and a shared virtual disk file in addition to the OS drivers.

The Basic Concepts

Now why is a clustered file system important? So basically if you have the need to have a shared volume between two hosts, you can provision the disk to both machines, and everything could appear to work, however in the event that writes ever happened to the same areas of the disk at the same time you will end up with data corruption. Now the key here is that you need a way to track locks from multiple nodes. This is called a Distributed Locking Manager or DLM. Now to get this DLM functionality working then it will create a cluster. Valid cluster nodes can then mount the disk and interact with it as a normal disk. So as part of OCFS2 we have two file systems which are created /sys/kernel/config and /dlm the prior is used for the cluster configurations, and the latter is for the distributed lock manager

Linux KVM: Ubuntu 12.10 with Openvswitch

Wed, 24 Oct 2012 00:00:00 +0000

Today I am revisiting my previous post on Openvswitch on Ubuntu 12.04. Things have changed since then. Previously Openvswitch was relatively new and as such the userland tools (with libvirt being the one I use) didn’t support it yet, so while you could have used Openvswitch by executing the kvm processes for each VM manually. Since this was a predictable problem the Openvswitch guys created a brcompat module, which would take the place of the bridge_utils functionality and expose Openvswitch bridges as legacy bridges. This allowed the existing userland which already had the legacy bridge functionality builtin to continue to operate the same way. Anyways as of libvirt 0.9.11 the native Openvswitch functionality is present and supported. As such I am updating my documentation to include this alternate and preferred method of operation.

Linux-LVM: Resize Partition to Grow LVM Volume Group

Mon, 20 Aug 2012 00:00:00 +0000

Logical Volume Manager makes the dynamic expansion of file systems dead stupid simple. However there is a weakness, if you are using a partitioned file system as your Physical Volume (PV) then you will end up needing to expand the file system if you ever need to grow the actual storage. This can be avoided by using the actual physical device /dev/sda as the PV, however if this is the same volume where your /boot partition lives then you will have no choice but to partition it out since /boot cannot reside on a Logical Volume (LV). The other way you can avoid this is by simply creating a second partition on the same disk and adding it as a new PV to the existing Volume Group (VG). This is preferable as there is far less room for error than in this procedure, but I personally find that method a little more confusing to inherit then a straight forward single partition made an LV.

Bash: Automatically Mount File Systems on Volume Group if Present

Thu, 02 Aug 2012 00:00:00 +0000

Update 09/16/2015 – I have published an updated article for this here: http://blog.allanglesit.com/2012/08/bash-automatically-mount-file-systems-on-volume-group-if-present/

In my laptop I have a solid state disk, and frankly I am way past addicted to solid state disks, but what you get in performance you lose in capacity. As such I have had to be creative with how I can have the capacity and the performance that I need. Lately the solution has been to have a spinning disk in the slot that would normally be occupied by my CD/DVD drive. However the problem with this, revealed itself to be that my Fedora install would actually fail to boot when it tried and failed to mount the file systems if this device was not present. So my first “solution” was to set noauto in the /etc/fstab for these secondary file systems. Which presented a new problem… I would forget to manually mount these devices before launching Virtualbox to start my VMs. So I decided that I would need to have an auto-mount script which would check for the volume group to be present and then if it was mount the file systems contained within the /etc/fstab which live on the volume group.

LVM: Using 2TB Volumes for Volume Groups

Mon, 28 May 2012 00:00:00 +0000

Occasionally you will have the need to use physical volumes which exceed the limits of the standard DOS partitioning scheme. When you run into this a quick fdisk -l will reveal that you will need a new way to partition your disks.

Fdisk will produce output similar to this…

 text
 
 
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

WARNING: The size of this disk is 2.3 TB (2290828181504 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID
partition table format (GPT).

Use Parted to Create a Partition Using the Whole Disk

Parted is able to handle large volumes which are not able to be modified with fdisk. Here we are going to launch parted, and then create the partition with a size of 1 -1 which means that it will use it all until it gets to the end. You can additionally specify the number in megabytes, so 1 1024 would start at 1 and go until 1024 giving you a 1GB partition.

Oracle VM: Using Oracle VM Manager 3 from Ubuntu Linux

Wed, 16 May 2012 00:00:00 +0000

In my initial work with Oracle VM, I have run into a few fringe cases, which are not well documented, but actually have rather simple solutions. This one actually cropped up over the weekend for me. Over the weekend I decided to rebuild my company-issued laptop with Ubuntu Linux (12.04 amd64) to be specific. I did this for a number of reasons…

1) I am far more effective with Linux, due to the nature of the shell, and tools which I am familiar with (vim, dd, really take your pick they are everywhere).

Oracle VM 3.x: Reset Agent Password

Tue, 15 May 2012 00:00:00 +0000

Oracle VM, is a centrally managed hypervisor. All administrative actions must be performed from the Oracle VM Manager. In order to facilitate the initial discovery of the Hypervisor by the Management Server part of the setup requires that we not only set the password for the root user, but also the password for the ovs-agent. We then initiate a discovery process from the management server to the hypervisor and end up able to manage our hypervisor.

Installing Oracle VM Manager 3.1.1

Mon, 14 May 2012 00:00:00 +0000

In my new role at Keste, I am tasked with staying on the bleeding edge of software release cycles, so that we can be knowledgeable and authoritative about technology when we engage with customers. One of the areas where Keste leads is on Oracle’s Virtualization Offerings. So I will be posting more and more content with respect to those offerings.

Today I will be documenting the steps to install Oracle VM Manager 3.1.1. Oracle VM (OVM) is a Xen-based Virtualization Technology. An OVM Deployment consists of the OVM Server (the bare metal hypervisor) and the OVM Manager, the application stack which manages the afforementioned hypervisors (organized into Server Pools).

Bash: A Better Way to Sed

Tue, 01 May 2012 00:00:00 +0000

So I frequently find myself in a situation where I need to modify files on a fairly programmatic basis, so sed has become a friend of mine for a lot of these situations. So lets start with some basic sed…

If I have a file..

 bash
 
 
cat /etc/file.txt
blah
blah
teststring
blah
blah

And I want to replace “teststring” with “productionvalue” then I could use sed.

bash

sed -i 's/teststring/productionvalue/g' /etc/file.txt

 bash
 
 
cat /etc/file.txt
blah
blah
productionvalue
blah
blah

This is all well and good, but recently I found a better way…

Bash: Duplicate Logical Volume Configuration for a Volume Group

Mon, 30 Apr 2012 00:00:00 +0000

I recently ran into a situation where I needed to create a large number of Logical Volumes for a server replacement situation. I could have done some manual data input, but why not take the opportunity to quickly write a script to do the hard work for me?

For my situation this was a server that pulls data from other servers in order to optimize getting that data to tape. So we had 2 VGs, one for the system and one for the data we wanted to move. We planned on simply performing a rsync to move the data itself, but the volumes had to be pre-staged.

Preparing Minimal Images with Debootstrap

Tue, 27 Mar 2012 00:00:00 +0000

Debootstrap gives us a simple and consistent way to cleanly setup machines by installing packages directly into a file system directly from the repos. These have the flexibility of being used across many different kind of machines.

Create Directory Structure

Here we are just creating the directories into which we will be installing the operating systems into.

 bash
 
 
mkdir -p /mnt/build/amd64 /mnt/build/i386
cd /mnt/build/amd64; mkdir ubuntu_10.04 ubuntu_10.10 ubuntu_11.04 ubuntu_11.10 debian_5 debian_6
cd /mnt/build/i386; mkdir ubuntu_10.04 ubuntu_10.10 ubuntu_11.04 ubuntu_11.10 debian_5 debian_6

Install Required Software

bash

apt-get install debootstrap

Install Operating Systems into the Folders

Ubuntu Lucid

Linux-KVM: Ubuntu 12.04 with Openvswitch

Mon, 26 Mar 2012 00:00:00 +0000

UPDATE: October 29, 2012

This article doesn’t work on Ubuntu 12.10, to use Ubuntu 12.10 please refer to my new article here.

Openvswitch is a better way of managing your virtual networking stacks for KVM and Xen. This can be used instead of the bridge-utils package, and has the ability to use VLANs, LACP, QoS, sFlow, and others. In this article we will be using Ubuntu 12.04 (beta1) to configure Openvswitch to hand out networking interfaces to KVM guests.

Securing SSH with Publicly Accessible Servers

Thu, 16 Feb 2012 00:00:00 +0000

Just about every IT environment has some sort of remotely managed environment which requires that they have SSH open to the Internet. Perhaps this is a VPS, dedicated server, or colocation. Regardless of your reason, the fact is that there is just some times where you need to have SSH open to the internet.

However regardless of if it is necessary doesn’t mean that you should just do it… One look at your auth.log will reveal that.

Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes Redux

Mon, 07 Nov 2011 00:00:00 +0000

This really is an update to my previous article “Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes“, but the process is significantly different and thus requires more explaining than a simple inline update would do it justice. Now the big flaw with my first article was that by using dd on the hypervisor level, we are opening ourselves up to unintentional data loss due to incompetence or even a very competent slip of the finger.

Bash: Using Temporary SSH Keys Within A Script

Wed, 10 Aug 2011 00:00:00 +0000

Every once and a while you will have the need to make multiple ssh calls to remote systems within a bash script. Normally you would simply have the user enter their password multiple times, or have a requirement for SSH keys to be configured prior to running the keys. This is widely accepted as the way to do it, however if that were the best way to do it, then I would not be writing this article. Now the better way to do it would be to create ssh keys, and upload them as authorized keys to the remote host(s) then upon completion of the work we can then clean up the changes that we made.

Bash: Using pv to Display Progress of dd

Tue, 09 Aug 2011 00:00:00 +0000

One of the biggest weaknesses of dd is that it has no way to display progress on its actions. You can send a signal to the process which will pause, display statistics, and resume the process however this takes up alot of your terminal screen if you are doing any sort of long running copy. Enter pv. Pv allows us to monitor the progress of data through a pipe.

Create A 1GB Test File

This is the file that we will use to peform the following tests.

Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes

Wed, 09 Mar 2011 00:00:00 +0000

Linux Logical Volume Manager provides a better method for managing your disks in the home and even more so in the datacenter. I briefly outlined some of the benefits afforded us by LVM2 in a previous article Linux LVM2: Flexible Local Storage Management. In this article I will provide you with some very simple steps and scenarios to migrate your existing KVM VMs to LVM2 Logical Volumes.

In this article I am going to make an assumption that your disk images are raw disk images and that you are not using qcow2, vhd, vdi, vmdk or any other disk image format. Additionally this process should work for a physical or RAID device as well, though I have not verified that.

Linux LVM2: Flexible Local Storage Management

Tue, 01 Mar 2011 00:00:00 +0000

Linux LVM provides a different way of looking at storage. On the server side this provides much more flexibility than standard disks with partitions. It allows us to migrate from old to new storage as well as expand the available storage to a particular file system easily (assuming that there is free space available).

Basically LVM abstracts the physical storage by creating Physical Volumes (PV), and each PV is divided into Physical Extents (PE) which are a consistent size. Physical Volumes are then added to a Volume Group (VG) which essentially pools all of the PE on all member PV. Once you have a VG you can then create a Logical Volume (LV) which can be formatted and used as if it were a physical disk, the difference is that the LV can be expanded as many times as needed up until all PE have been allocated within the VG. In this event you could then simply add another PV to the VG, which would make available additional PE to be added to a LV.