the bin

Amazon VPC Basics

Mon, 06 Aug 2018 00:00:00 +0000

For the past five weeks we have been getting deep in our Scalable Solutions series. At Entasis we work with our customers to help them build solutions that are Scalable in design, Resilient by nature, with Performance on demand. We feel that these three components are the pillars that all truly scalable solutions are built with. Here is what we have gone over thus far:

Week 1 – Amazon Route 53 Basics

Amazon EC2 Security Groups

Mon, 30 Jul 2018 00:00:00 +0000

In this installment of our Scalable Solutions series we are going to be reviewing one of the core components of EC2, the security group. We have found that our customers find that the elasticity provided by security groups allows them to build more resilient solutions and expand them as performance dictates it, with consistent security policies.

In case you missed our previous articles you may want to go back and review these great resources.

AWS Application Load Balancer

Mon, 23 Jul 2018 00:00:00 +0000

This week in our Scalable Solutions series we are going to be covering the Elastic Load Balancer (specifically the Application Load Balancer or v2). At Entasis we help our customers build elastic and resilient solutions with performance on-demand. If you need help finalizing your design, or implementing your vision please don’t hesitate to reach out to us at sales@entasistech.com. Also if you are just looking to learn more about AWS and you are in the North Dallas area we sponsor the AWS User Group of North Dallas at our offices in McKinney, TX. You can find this on meetup.com.

AWS Systems Manager Parameter Store

Mon, 16 Jul 2018 00:00:00 +0000

At Entasis we help our customers understand and modify their technology stacks so that scaling is a function of cost not a function of technology. However when we look at scaling one of the largest problems is the consistency of an environment. When we are running 100 application nodes then the opportunity for one node to go sideways and start handling traffic differently is much higher than if we only have 2 application nodes. So we work with our customers to help them abstract the configurations of their services from their code to enable their applications to auto-scale regardless of your hosting location. We are very familiar with public cloud, private cloud, multi cloud and hybrid cloud. If you need help in this area please reach out to our team at sales@entasistech.com.

AWS Certificate Manager

Mon, 09 Jul 2018 00:00:00 +0000

Today we are going to take a look at another one of my favorite services. AWS Certificate Manager. In this exploration we are going to be interacting with JSON which Powershell makes more difficult then it ought to. So please keep in mind that if you are using the AWS CLI on bash or cygwin or similar your mileage will vary (especially where it comes to parsing and generating JSON). Our goal is to be able to walk all the way through the certificate issuance process, including validation.

Amazon Route 53 Basics

Sun, 01 Jul 2018 00:00:00 +0000

This post will kick off a series on Scalable Solutions. Entasis uses a strong architectural emphasis in our AWS Solutions Group to deliver solutions that are Elastic by nature, Resilient in design, with Performance on demand. These are the three pillars of Scalable Solutions. In this series we will be going through some of the architectural components and decisions which allow us to design and deliver all three of these pillars in our solutions.

Linux: Replace Disk in MD

Mon, 28 Dec 2015 00:00:00 +0000

On my lab machines I use a pair of physical disks, configured in a RAID 1 with MD, with LVM on top of that. This gives me a lot of flexibility with full redundancy. However recently I had a failure of a drive.

Identify the Failure

Here is what MD was telling me after I identified and removed the drive from the chassis to send it off for an RMA, notice how each device only has one member (md0 has sda1, md1 has sda2), this does not make a good RAID 1. But this is easily resolved.

Bash: Dynamic Volume Group Mounter

Wed, 16 Sep 2015 00:00:00 +0000

This is similar, but an enhanced version of my custom_mount.sh previously published here: Bash: Automatically Mount File Systems on Volume Group if Present. This previous version was just a simple init script. This updated version is designed for newer Linux distributions that use Systemd instead of Init.

The Code

The majority of the work is done by dvgmounter.sh. It will check if the Volume Group is present, if it is it will parse the /etc/fstab file looking for entries that are marked noauto, then it will test if that file system is already mounted, if it is not it will mount it.

Fedora 20: Use PPTP VPN

Tue, 15 Sep 2015 00:00:00 +0000

I have had this one in the queue for a while (I don’t even run of Fedora 20 anymore). This one will be short and sweet, because there really isn’t a bunch of discussion that needs to go into it.

By default Fedora 20 will block outbound PPTP VPN connections.

 bash
 
 
firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -p gre -j ACCEPT
firewall-cmd --direct --add-rule ipv6 filter INPUT 0 -p gre -j ACCEPT

Reload the rules.

Zimbra 8.6: Bayesian Poisoning

Mon, 14 Sep 2015 00:00:00 +0000

Let me start by saying that this problem is not unique to Zimbra and it certainly isn’t unique the version 8.6, however I was using Zimbra 8.6 when I ran into this problem, so this is how I fixed it.

What is Bayesian Poisoning?

One of the core tenants of spam filtering using Bayesian probability to increase or decrease a particular messages score based on the likelihood it is spam. This is done by compiling a database, often called the Bayes DB, which contains tokens resulting from the Bayesian filtering, these tokens are keywords and combinations that will either push up or down the probability that a given message is spam. So Bayesian Poisoning is when that DB is intentionally populated with invalid references which result in either more spam being marked as not spam, or more legitimate mails being marked as spam.

Fedora 20: Firefox Reports Flash as Vulnerable

Wed, 04 Mar 2015 00:00:00 +0000

This problem starts with Firefox reporting that your flash-plugin is out of date. This report looks like this and disables all flash.

After this we will take a look Mozilla’s Plugin Check to see what it thinks is going on.

Now here we can see that version 11.2.202.440 is vulnerable. We will then check about:plugins to see if it agrees.

Again this is also reporting 11.2.202.440, so there must be a problem, but it also tells us that there is an update available. Now I run regular yum updates on this machine, and I actually noticed flash-plugin was updated just a few hours prior to seeing this alert. So lets check the installed version.

BlackBerry Z30: No Update to 10.3.1

Thu, 26 Feb 2015 00:00:00 +0000

I have a BlackBerry Z30 (STA100-5) which I was excited to updated to the latest release of BB10, which was announced on February 9, 2015 (link). However when I was attempting to install the update over the air, it kept telling me that I was already on the latest version. This was obviously incorrect (I was on 10.2.1.3062 which is the latest prior to 10.3.1).

Here are the things that I tried that were unsuccessful.

SPARC Logical Domains: Alternate Service Domains Part 3

Wed, 17 Dec 2014 00:00:00 +0000

In Part One of this series we went through the initial configuration of our Logical Domain hypervisor, and took some time to explain the process of mapping out the PCI Root Complexes, so that we would be able to effectively split them between the primary and an alternate domain.

In Part Two of this series we took the information from Part One and split out our PCI Root Complexes and we configured and installed an alternate domain. We were also able to reboot the primary domain without impacting the operation of the alternate domain.

SPARC Logical Domains: Alternate Service Domains Part 2

Tue, 16 Dec 2014 00:00:00 +0000

In Part Two (this article) we are going to take that information and split out our PCI Root Complexes and configure and install an alternate domain. At the end of this article you will be able to reboot the primary domain without impacting the operation of the alternate domain.

SPARC Logical Domains: Alternate Service Domains Part 1

Mon, 15 Dec 2014 00:00:00 +0000

In this series we will be going over configuring alternate I/O and Service domains, with the goal of increasing the serviceability the SPARC T-Series servers without impacting other domains on the hypervisor. Essentially enabling rolling maintenance without having to rely on live migration or downtime. It is important to note, that this is not a cure-all, for example base firmware updates would still be interruptive, however minor firmware such as disk and I/O cards only should be able to be rolled.

SPARC Logical Domains: Live Migration

Tue, 09 Dec 2014 00:00:00 +0000

One of the ways that we are able to accomplish regularly scheduled maintenance is by utilizing Live Migration, with this we can migrate workloads from one physical machine to another without having service interruption. The way that it is done with Logical Domains is much more flexible than with most other hypervisor solutions, it doesn’t require any complicated cluster setup, no management layer, so you could literally utilize any compatible hardware at the drop of the hat.

Solaris 11: Configure IP Over Infiniband Devices

Mon, 08 Dec 2014 00:00:00 +0000

In this article we will be going over the configuration of an infiniband interface with the IPoIB protocol on Solaris 11, specifically Solaris 11.2 (previous versions of Solaris 11 should work the same, however there have been changes in the ipadm and dladm commands).

Identify Infiniband Datalinks

First we need to identify the underlying interfaces of the infiniband interfaces. In my case net2 and net3.

 bash
 
 
dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown ixgbe1
net0 Ethernet up 1000 full ixgbe0
net2 Infiniband up 32000 unknown ibp0
net3 Infiniband up 32000 unknown ibp1
net5 Ethernet up 1000 full vsw0

Another way to confirm the infiniband interfaces is to use the show-ib command.

Adventures in ZFS: Mirrored Rpool

Mon, 01 Dec 2014 00:00:00 +0000

It always makes sense to have a mirrored rpool for your production systems, however that is not always how they are configured. This really simple procedure is also critical.

Create a Mirrored Zpool

Check the existing devices to identify the one currently in use.

 bash
 
 
zpool status rpool
 pool: rpool
 state: ONLINE
 scan: none requested
config:

 NAME STATE READ WRITE CKSUM
 rpool ONLINE 0 0 0
 c0t5000CCA0436359CCd0 ONLINE 0 0 0

errors: No known data errors

Once we know which one is currently in use, we need to find a different one to mirror onto.

Linux KVM: Bridging a Bond on CentOS 6.5

Mon, 27 Oct 2014 00:00:00 +0000

Today we are going to hop back into the KVM fray, and take a look at using CentOS as a hypervisor., and configuring very resilient network connections to support our guests. Of course these instructions should be valid on Red Hat Linux and Oracle Linux as well, though there is a little more to be done around getting access to the repos on those distributions…

Enable Bonding

I am assuming this is a first build for you, so this step might not be applicable, but it won’t hurt anything.

BlackBerry OS 10: Caldav Setup with Zimbra

Tue, 21 Oct 2014 00:00:00 +0000

I have owned my Blackberry Z10, going on a year now, and I have absolutely loved it. However one of the issues that I have fought was in integrating it with my Zimbra Installation. Email was easy, the IMAP protocol sorted that out easily enough… However calendars turned out to be more of a challenge than I expected.

Here is the versions that I validated these steps on.

Blackberry Z10 with 10.2.1.2977
Zimbra Collaboration Server 8.5.0

Here is how to get it done.

Adventures in ZFS: Splitting a Zpool

Wed, 09 Jul 2014 00:00:00 +0000

Today we are going to go through the process of splitting a zpool. This is not something that you will do often, but when you do it, you must be very precise and do it properly.

Create Our Test Zpool

First we will create a ZFS to hold our disk images.

bash

zfs create rpool/testdisks

Next we will create the disk images.

 bash
 
 
mkfile 1g /rpool/testdisks/disk0
mkfile 1g /rpool/testdisks/disk1
mkfile 1g /rpool/testdisks/disk2
mkfile 1g /rpool/testdisks/disk3

Finally we will create our zpool, which in this case is tank.

Adventures in ZFS: Destroy A Zpool Without Importing

Tue, 01 Jul 2014 00:00:00 +0000

Recently I ran into an issue where I had a zpool that wasn’t imported in a system, and I was unable to import it due to zpool version incompatibilities. This can happen if you are going to a older revision of Solaris or if you are crossing platforms between ZFS and OpenZFS. This posed an interesting problem for me, as how do I destroy a pool that I haven’t imported? There might be a quick command to handle this, but this actually presented another problem, as in my example I actually had a zpool name collision as well (this was the second mirror in an old rpool).

Install MySQL Server Community Edition on Fedora 20

Tue, 01 Jul 2014 00:00:00 +0000

Today we are going to go through the process of installing MySQL Server Community Edition on Fedora 20. You could of course simply use MariaDB, but this will focus on the Community Edition.

Install the MySQL Repository

We will need to install the MySQL Community Repository, the MySQL project hosts an RPM file with the repository. First lets look at the repositories that I have already.

 bash
 
 
yum repolist
Loaded plugins: langpacks, refresh-packagekit
repo id repo name status
adobe-flashplayer Adobe Flash Player - x86_64 2
fedora/20/x86_64 Fedora 20 - x86_64 38,597
google-chrome Google Chrome - x86_64 3
oracle-virtualbox/20/x86_64 Oracle VirtualBox - x86_64 10
rpmfusion-free/20/x86_64 RPM Fusion for Fedora 20 - Free 468
rpmfusion-free-updates/20/x86_64 RPM Fusion for Fedora 20 - Free - Updates 464
rpmfusion-nonfree/20/x86_64 RPM Fusion for Fedora 20 - Nonfree 203
rpmfusion-nonfree-updates/20/x86_64 RPM Fusion for Fedora 20 - Nonfree - Updates 299
updates/20/x86_64 Fedora 20 - x86_64 - Updates 17,616
repolist: 57,662

Now lets install the release file. Keep in mind, you can do a separate wget or curl to download the file, or you can use the command as I have. As of the time of this writing there are also repositories for Fedora 18 and 19 and Enterprise Linux 5, 6 and 7 (beta). Though the only procedures I have validated are the Fedora 20, they should all work the same.

SQL Developer Crash on Fedora 20

Tue, 01 Jul 2014 00:00:00 +0000

I ran into a painful issue on Fedora 20 with SQL Developer. Basically every time it was launched via the shortcut it would go through loading, and then disappear.

Manual Invocation of SQL Developer

When launching it via the script itself it gives us a little more information.

 bash
 
 
/opt/sqldeveloper/sqldeveloper.sh

Oracle SQL Developer
Copyright (c) 1997, 2013, Oracle and/or its affiliates. All rights reserved.

 

LOAD TIME : 279#
A fatal error has been detected by the Java Runtime Environment:
#
SIGSEGV (0xb) at pc=0x00000038a1e64910, pid=12726, tid=140449865832192
#
JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 1.7.0_40-b43)
Java VM: Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode linux-amd64 compressed oops)
Problematic frame:
C 0x00000038a1e64910
#
Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
An error report file with more information is saved as:
/opt/sqldeveloper/sqldeveloper/bin/hs_err_pid12726.log
[thread 140449881597696 also had an error]
#
If you would like to submit a bug report, please visit:
http://bugreport.sun.com/bugreport/crash.jsp
#
/opt/sqldeveloper/sqldeveloper/bin/../../ide/bin/launcher.sh: line 611: 12726 Aborted (core dumped) ${JAVA} "${APP_VM_OPTS[@]}" ${APP_ENV_VARS} -classpath ${APP_CLASSPATH} ${APP_MAIN_CLASS} "${APP_APP_OPTS[@]}"

I also noticed, that while executing as root it worked. However that clearly isn’t the “solution”

Voyage Linux: Dialog Error with Apt

Tue, 01 Jul 2014 00:00:00 +0000

This can happen on other Linux distributions, however in this case I found it on Voyage Linux, which is a Linux distribution for embedded hardware.

The Error

Here we are dealing with an annoyance whenever you use apt-get or aptitude.

 text
 
 
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)
debconf: falling back to frontend: Readline

The Fix

Simply install dialog, which is the package it is not finding. This will no longer need the failback to readline.

Voyage Linux: Locale Error with Apt

Tue, 01 Jul 2014 00:00:00 +0000

Voyage Linux is an embedded linux distribution. I use it on some ALIX boards I have lying around, it is very stripped down, and as such there are a few annoyances which we have to fix.

The Error

This issue happens when attempting to install/upgrade packages using apt-get or aptitude.

 text
 
 
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LANG = "en_US.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

The Fix

Adventures in ZFS: Rename a Zpool

Mon, 30 Jun 2014 00:00:00 +0000

Renaming a zpool is not as intuitive as I would expect, but it is still pretty simple. We accomplish this through a three step process, export the zpool, import the zpool (using a new name), validate our mountpoints are how we want them (probably using the new zpool name).

Now why might I want to rename it? That is simple, perhaps it is simply for organization or perhaps you already have a zpool by that name, or perhaps you use zpool mirrors as a method of backup (break the mirror rename one side, rebuild mirror with a new disk maintaining the previous copy offline).

BIND 9: Configure Slave Zones

Wed, 25 Jun 2014 00:00:00 +0000

Today we are going over one of the fundamental aspects of administering a DNS infrastructure, that is maintaining consistency across multiple replicas. The simplest method of accomplishing this is to have a master/slave relationship. Essentially you have a read-write (master) copy and a read-only (slave) copy of the zone. The slave looks to the master for information on changes.

Basic BIND Configuration

Here we are going to look at the key components of a BIND configuration, this should be the same regardless of if this is a master or a slave server.

Codespaces.com and DevOps: Why We Should Care

Sun, 01 Jun 2014 00:00:00 +0000

Generally on this blog I tend to be far more technologist than pundit, in that I normally don’t write about something unless it has to do with doing something interesting technically. However in this case I think a bit of talk about what happened and why might be in order for the architecture minded out there.

What Happened?

Let me first start by saying that I have no inside information here, I have never been a customer of codespaces.com, and have never in any capacity performed any work for them. The information I have is all pulled from the Google Cache of their site here. Now according to Codespaces.com on Tuesday June 17, 2014 they had a DDOS, and it appears that as a result of this DDOS attack that an attacker was able to gain access to their EC2 control panel. The attackers ultimate goal was extortion, however when Codespaces attempted to regain control the attacker still had access to the control panel and was able to start deleting backups, storage and instances either strategically or indescriminately. Now keep in mind, these are the current theories about what has happened, it could turn out that the attacker already had access prior to the DDOS attack, perhaps from a previous attack, and they were just planning their next move.

Solaris 11: Install Bind Server

Sun, 01 Jun 2014 00:00:00 +0000

Today I am going to document the process of install BIND in Solaris 11. I am using a Solaris 11.1 zone for this task, though nothing in this is specific to zones, and it should work on previous versions of Solaris 11. I have done this quite a few times and it is not a very intuitive process. As of the time of this writing the version of BIND in IPS is 9.6.3.7.2 (9.6-ESV-R7-P2).

Solaris Networking: Datalink Multipathing Protocol

Mon, 19 May 2014 00:00:00 +0000

In this article we are going to go over Datalink Multipathing Protocol (DLMP) available in Solaris. DLMP is similar to IPMP, however there are some key differences, the biggest being the layer at which it operates. DLMP operates at the datalink layer of the OSI model, while IPMP operates at the network layer. Due to the differences in those layers DLMP opens up a lot of possibilities that were not possible with IPMP. For example if you had a requirement for redundant networking for a service, either IPMP and DLMP would both be able to meet that requirement very well. However if you had a requirement that the service runs inside of a zone or a logical domain then the level of work in IPMP becomes much higher. This is because these hypervisors assign datalink devices to their guests. Since the redundancy for IPMP is built on a level higher we need to assign multiple non-redundant interfaces to the guests and then build the interfaces and IPMP groups inside each of the guests.

Solaris SMF: Define File Dependency

Thu, 15 May 2014 00:00:00 +0000

One of the biggest benefits of migrating to the Service Management Framework is that we can introduce dependencies to a services, these dependencies can be other services which is very valuable, however they can also be file based. This can be used in a number of ways, say for example you have some application trees that live on some NFS mounts, in the event that the mount hasn’t been successfully mounted it will still attempt to start the service. However if we make our service dependent on the file, then if the file doesn’t exist it won’t even attempt to start the service.

Solaris SMF: Define Service Execution User

Wed, 14 May 2014 00:00:00 +0000

Frequently we want a service to execute as a non-root user, this is pretty trivial in the context of the Service Management Facility in Solaris. This article will go over exactly what goes into that. The one complexity around that is the environment, if you have environment variables that the user will depend on those, they will need to be setup inside of the SMF. I will not be going into that in this article.

Solaris SMF: Delegate Service Control

Tue, 13 May 2014 00:00:00 +0000

In many of the environments that I deliver to a client we have a concept of an “environment account” this account is the credential set that the environment runs as. Over time and the maturation of the Service Management Facility (SMF) we have been able to simplify this offering and increase its capabilities. Today we are going to talk about allowing the environment user to manage a service controlled by SMF.

ZFS: Send and Receive

Tue, 24 Sep 2013 00:00:00 +0000

ZFS gives us the ability to move data with ZFS send and receive. This can be used to combine snapshots, create full or incremental backups, or replicate data between servers on your LAN or even over the WAN.

Local Backup with Send

This is very simple, simply perform a send of a snapshot then output it to the file, I have complicated it a bit to include a folder which is named off of the current date (as would be helpful with backups).

Solaris Networking: IP Multipathing Protocol

Tue, 10 Sep 2013 00:00:00 +0000

In this article we are going to go over some basics of using IPMP in Solaris. We will discuss this on the latest versions of Solaris 10 and 11 on the SPARC platform, these instructions should also work on the x86 platform, but will be dependent on the listed version numbers, as the networking stack has changed alot of the past few minor versions of Solaris 10 and 11.

Solaris 11.1

Create an IPMP Group

Adventures in ZFS: NFS Exports on Solaris 10 and 11

Sun, 01 Sep 2013 00:00:00 +0000

So with ZFS you really have two ways of configuring your NFS exports, you can do it the old fashioned way (read: /etc/dfs/dfstab and share command). Or the super-duper easy method… Use ZFS metadata, and allow ZFS to configure the export for you. The reason why you might choose this approach is simple, if you don’t want to have to worry about configs, and especially if you need this export to be tied to the filesystem regardless of where you put the file system from system to system. Pause for effect. Your NFS exports can be TIED TO THE VOLUME AND NOT THE SERVER. So if you have a USB drive that you are using to move data on, it can immediately share the data on another server when the pool is imported, same is true for LUNs. This adds a lot of flexibility.

Adventures in ZFS: Solaris 11.1 Share Properties

Sun, 01 Sep 2013 00:00:00 +0000

In Solaris 11.1 Oracle decided to change the way that they stored filesystem share information. I personally loved the change, but that was without understanding what the change really was. For more information on how to enable NFS exports see my previous article here.

Setup Our Environment

Everything we are doing will be on some test file systems, with no data. So lets put everything in place.

 bash
 
 
zfs create tank/fs1
zfs set share=name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15 tank/fs1
name=tank_fs1,path=/tank/fs1,prot=nfs,sec=sys,rw=@10.0.0.15

Now lets create our child filesystem, and export it the same way.

Keste is Hiring

Thu, 01 Aug 2013 00:00:00 +0000

Generally I don’t use my blog as a platform to find potential applicants for my employer, however at this point we are looking very seriously at some new candidates to fill out our Systems Team.

This is a job that is very much “Systems” focused. At Keste we define Systems as everything from the hardware up to the applications, this includes but is not limited to Servers, Storage, Networking and Virtualization. Because this is a very broad area, we expect folks to have expertise in multiple areas and most importantly to have the capabilities to learn and broaden their knowledge where they have weaknesses.

Powerconnect: Configure Link Aggregation Groups

Tue, 16 Jul 2013 00:00:00 +0000

Today we will be configuring LACP to interconnect some switch stacks as part of a network upgrade. Please keep in mind that part of configuring the LAG will result in the ports losing connectivity, so while it is not necessary to use a console cable, please keep in mind how you are connecting to the switch and if you will interrupt that connection by any of your actions, and plan accordingly.

Dell Powerconnect 6200: Enable SSH and Disable Telnet

Mon, 08 Jul 2013 00:00:00 +0000

Most modern switches include the ability to use SSH as a remote communications protocol. Here we will enable that functionality and disable telnet on the Dell Powerconnect 6248 switches.

Enable SSH

In order to enable SSH we need to generate the keys which will be used by SSH to encrypt the traffic.

 text
 
 
console>enable
console# configure
(config)# crypto key generate rsa
(config)# crypto key generate dsa
(config)# ip ssh server
(config)# exit
console# copy running-config startup-config

Disable Telnet

Depending on your config you might not need to disable this, but if it is enabled you should disable it.

Dell Powerconnect 3500: Enable SSH and Disable Telnet

Mon, 01 Jul 2013 00:00:00 +0000

Most modern switches include the ability to use SSH as a remote communications protocol. Here we will enable that functionality and disable telnet on the Dell Powerconnect 3548 switches.

Enable SSH

In order to enable SSH we need to generate the keys which will be used by SSH to encrypt the traffic.

 text
 
 
console>enable
console# configure
(config)# crypto key generate rsa
(config)# crypto key generate dsa
(config)# ip ssh server
(config)# exit
console# copy running-config startup-config

Disable Telnet

Dell Powerconnect 5500: Enable SSH and Disable Telnet

Mon, 01 Jul 2013 00:00:00 +0000

Most modern switches include the ability to use SSH as a remote communications protocol. Here we will enable that functionality and disable telnet on the Dell Powerconnect 5548 switches.

Enable SSH

In order to enable SSH we need to generate the keys which will be used by SSH to encrypt the traffic.

 text
 
 
console>enable
console# configure terminal
(config)# crypto key generate rsa
(config)# crypto key generate dsa
(config)# ip ssh server
(config)# exit
console# write memory

Disable Telnet

Fedora 18: Noticing Clock Skew?

Mon, 01 Jul 2013 00:00:00 +0000

Recently I noticed that my clock was beginning to skew, and NTP wasn’t able to keep up with it. This is due to having a “system clock” and a “hardware clock” the system clock is the one that the OS controls, while the hardware clock is controlled by the BIOS and runs on the hardware. So all I did to resolve it was sync up the system clock and then sync the hardware clock based off of the system clock.

Adventures in ZFS: Deletion of a Recursive Snapshot Set

Mon, 03 Jun 2013 00:00:00 +0000

Quite often I will find it necessary to create a recursive snapshot over a large number of ZFS file systems. In this case, I find that frequently have little long term need for them, and need a quick and easy way of disposing of the snapshots created without spending large amounts of time manually deleting all of the snapshots.

Creating Some Snapshots

Here we will create some test snapshots. Make sure you do this on a test system, in a throw away area, this way if something is mistyped it doesn’t cause any problems.

Media Management: Rename JPG Images Based on Metadata

Sat, 01 Jun 2013 00:00:00 +0000

I have been doing a bit of cleanup around my home file server, and have noticed that while I have a really beautiful collections of family photographs there is no logical organization around these photos. I have many duplicates names, which would collide in the event of trying to merge multiple directories, and even worse I have many duplicate copies of pictures from various uploads from various devices.

Today we are going to focus on one aspect of that problem, we are going to rename our JPG images based on when the photo was taken. This will require that the camera which took the photo was writing this metadata to the image in the EXIF format, or that you have manually coded the correct dates in them. Additionally it also requires that your camera have the correct time on it at the time the picture was taken. This isn’t so much an issue in the smart phone era but it was an issue with some of my older photos.

Oracle Linux 6: Create an OCFS2 Cluster and Filesystem

Mon, 04 Mar 2013 00:00:00 +0000

Today we are going to go through the process of creating a clustered file system on a pair of Oracle Linux 6.3 nodes. This exercise is not very resource intensive. I am using two VMs each with 1GB of RAM a single CPU and a shared virtual disk file in addition to the OS drivers.

The Basic Concepts

Now why is a clustered file system important? So basically if you have the need to have a shared volume between two hosts, you can provision the disk to both machines, and everything could appear to work, however in the event that writes ever happened to the same areas of the disk at the same time you will end up with data corruption. Now the key here is that you need a way to track locks from multiple nodes. This is called a Distributed Locking Manager or DLM. Now to get this DLM functionality working then it will create a cluster. Valid cluster nodes can then mount the disk and interact with it as a normal disk. So as part of OCFS2 we have two file systems which are created /sys/kernel/config and /dlm the prior is used for the cluster configurations, and the latter is for the distributed lock manager

Oracle VM 3: Connecting to MySQL Backend

Wed, 13 Feb 2013 00:00:00 +0000

Starting in Oracle VM 3.2.1 the built in database of the Oracle VM Manager was MySQL. I had hoped that this change would also signal a change in the database schema. In prior versions of OVM 3.x all data was populated in the database in a completely useless longblob form.

 bash
 
 
mysql ovs -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock'

As we can see it is attempting and failing to use /var/lib/mysql/mysql.sock as the connection. So lets take a look at the process and see if it has any clues.

MySQL: Describe All Tables

Fri, 01 Feb 2013 00:00:00 +0000

In MySQL if you want to find out what columns are in a given table, you can describe the table and it will show you the columns and the data types associated with that column. However if you need to do this for a large number of tables, it can get a bit repetitive.

Connect to MySQL

 bash
 
 
mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.1.37 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

Show the Available Databases

OVMCLI: Standardize the Case of VM Names

Fri, 01 Feb 2013 00:00:00 +0000

So I have been working with the OVM CLI lately to try and fill in some holes in my knowledge and provide for some easily reproducible procedures, usually around consistency of the environment when performing frequent tear downs and upgrades. So expect to see more OVM related scripts going forward.

Today we are going over a fairly simple script. I like to use all lower case in my VM naming conventions. So I wrote up a little script to check every name in the environment and then rename it if it is not already lower case. Then once I completed this I decided to change it to be a toggle, so that those of you who prefer upper case would not feel left out. Basically what we are doing, is simply performing a few calls to the ovmcli to collect the information necessary to make our changes. Then we perform the changes via a separate call. Prior to making the edit call, we compare the actual value with the requested value in order to see if it is already what we wanted.

OVMCLI: Standardize the Naming of Virtual Disks

Fri, 01 Feb 2013 00:00:00 +0000

Today we are going over a script I wrote to utilize the ovmcli to rename all virtual disks to follow a understandable standard, in this case we will take the vmname and append a disk identifier to it. The first disk (slot 0) will be appended with “_system.img” with all subsequent being appended with “_dataX.img” where X is the slot number. This is a pretty simple script, but depending on the size of your environment it could take a significant amount of time to do a run in your environment due to the number of calls that you need to make to the OVM Manager.

Solaris 11: Installing MySQL

Fri, 01 Feb 2013 00:00:00 +0000

Recently I have been spending some time learning about database technologies (Oracle Databases at Keste as well as MySQL on my own). Part of this I have decided to carry over into my existing work with Solaris, and go through the installation process using the Image Packaging System which is in Solaris. Now really the IPS does all the heavy lifting for us, but we still need to know how to utilize the package manager to get the desired result.

Oracle VM 3: Enable Key Based Authentication to OVM CLI

Tue, 01 Jan 2013 00:00:00 +0000

I have been spending some time learning the OVM 3 CLI since it was released in the earlier 3.x versions, however the biggest roadblock for me has always been that they only allow access via SSH on port 10000. However now I know how to enable key based authentication for these connections opening this up for a much more robust scripting experience.

Connect to OVM CLI with Password

 bash
 
 
ssh -p 10000 admin@ovmmgr.allanglesit.com
admin@ovmmgr.allanglesit.com's password:
OVM>

Authorize Keys for OVM CLI

Oracle VM 3: Orphaned VMs

Tue, 01 Jan 2013 00:00:00 +0000

One of my biggest complaints about Oracle VM 3 is that everything is abstracted at every level. A VM name is simply meta data stored in a database (kind of) while everywhere that it matters the VM only exists as a UUID, lets look at some examples.

The Way It Is

From the hypervisor if we use native Xen commands to look at all of the instantiated VMs.

 bash
 
 
xm list
Name ID Mem VCPUs State Time(s)
0004fb000006000002a00feb1f52cae4 155 8192 2 -b---- 39176.1
0004fb00000600000b5b1f8673c451be 171 8192 2 -b---- 1007276.5
0004fb000006000026a40b6fb53fe2c3 135 16384 2 -b---- 169716.7
0004fb0000060000506f591d6877c284 194 3072 1 -b---- 16108.4
0004fb00000600007052ad7b040749ea 176 8192 2 -b---- 1726118.2
0004fb0000060000a07b66923b0f68a4 138 4096 2 -b---- 2092394.0
0004fb0000060000abfcf710ebd640df 101 2048 2 -b---- 540488.4
0004fb0000060000bf16256b99159fc6 143 16384 2 -b---- 3972112.7
0004fb0000060000e3f234f6c1d26448 173 8192 2 -b---- 852019.6
0004fb0000060000e602b0e0969dba07 92 2048 2 -b---- 234244.0
0004fb0000060000f9205e11159e5459 175 12288 2 r----- 4145289.3
Domain-0 0 2486 24 r----- 4455022.6

Now since under the covers we are using Xen that means that somewhere there is a vm.cfg file (could be named anything, but for now they are sticking with vm.cfg).

Shore Up New Jersey: Hurricane Sandy Relief Efforts

Tue, 01 Jan 2013 00:00:00 +0000

I wanted to take a few minutes and let all of you know about an organization I had the pleasure and honor of working with recently. I have worked with similar organizations previously however there was something truly special about this organization. This organization was Heavenly Driven Disaster Relief website here.

About HDDR

Basically HDDR is a non-profit organization which want to help people afflicted by disasters, they have been all over the world working in disaster areas trying to help people clean up and rebuild their lives. At the center of this ministry is Albert Knight, he is a contractor with a passion for helping people. He coordinates volunteer teams of all skill levels to complete the work that is being done under the organization. While HDDR is a Christian Organization they do not require volunteers have an affiliation with a specific church or even religion. People of all faiths and lifestyles are encouraged to get involved. Another thing that HDDR does differently is they really try to defray some of the costs of your participation in the relief efforts. They are able to provide housing and food for groups, leaving you with only the costs associated with getting there.

Solaris 11: Defining A Service Using Svcbundle and the SMF

Tue, 01 Jan 2013 00:00:00 +0000

Lately I have been going through an exercise of migrating all of my current services into Solaris 11 (more specifically a zone). I have already been using similar technology on the Linux side, I use OpenVZ with a mix of Linux distributions. However I would like to take advantage of ZFS and get a little more simplicity in management. One of the most important hurdles to overcome when making the move from Linux to Solaris is simply the execution of services or processes. Now in Linux you would write a little init script and then set it to run at certain run levels by either using chkconfig or creating symbolic links into runlevel directories. For an example of what these init scripts would look like you can see my article “Bash: Automatically Mount File Systems on a Volume Group if Present“. Now I could simply port my script to run under Solaris (and use the existing rc.d symbolic linking which also exists), however this methodology only starts and stops services on a change of run level and it doesn’t actually do anything to ensure that these services stay up.

Solaris 11: Using Crashplan for Cloud Backups

Tue, 01 Jan 2013 00:00:00 +0000

I personally use Crashplan for my cloud based backups (I have for over a year). I use them because they are one of a few (to be honest the only one) who actually have Solaris installers. The one downside of this is that they do not start the CrashPlanEngine by default (as a service). However when using the Service Management Framework in Solaris this is really easy to handle yourself… For more details how to handle the SMF portion refer to my article “Solaris 11: Defining A Service Using Svcbundle and the SMF” which goes into much more detail on this subject.

Sudo: Setting Up Intelligent Security Policies Part One

Tue, 01 Jan 2013 00:00:00 +0000

What is Sudo?

Prior to sudo whenever you wanted to run a command as a specific user then you would su (switch user) to gain access to a shell for that user. This of course required that you knew the password for that user, and it would spawn a new shell after successful authentication. The major drawback to this was that in order to allow a user to do something, you had to allow the user to do anything. This sort of configuration also makes it fairly trivial for an authorized user to perform unauthorized actions, such as changing the root password and locking everyone out.

Sudo: Setting Up Intelligent Security Policies Part Two

Tue, 01 Jan 2013 00:00:00 +0000

In Part One we went over the basics of sudo, what it is, why we use it, and how it is used properly. In this article we are going to take it a step further and look at specific use cases for sudo. The key thing to remember though, is that you have two ways you can use sudo, 1) to allow for program execution as root, a white-list 2) to restrict program execution as root, a black-list. If you are doing the later there is no definitive way to restrict a program execution, so you should only use the latter with administrators who are trusted completely (the same users who would have access to the root password).

Windows 7: The Never Ending Printer

Tue, 01 Jan 2013 00:00:00 +0000

The past couple of weeks I have been dealing with a fairly aggravating issue at home. My wife has a Windows 7 laptop which for some reason started printing everything repeatedly (repeatedly being defined as forever and ever until the bottom of the paper tray was found). Our printer in particular is a HP Laserjet 1022n which is connected to the network, and my wife’s computer connects via the network directly (as in not using any print server to mediate the jobs). When this was occurring and looking at the print queue I was seeing the one job, however the job kept flashing (or perhaps blinking) and it seemed like the job was being continuously restarted. This would even happen for test pages. The only other machines in the house are Linux/Unix and none of them had the same issue so it didn’t appear to be caused by the printer itself. One other detail was that I purchased this printer new in 2005 and it has been working just fine since then, with this computer and others.

Google Galaxy Nexus: Android 4.2 Over The Air Update

Thu, 01 Nov 2012 00:00:00 +0000

The Problem

I own both a Google Galaxy Nexus (the phone) and a Google Nexus 7 (the tablet). Both of these devices have served me well in the time I had them. Until the Over-the-Air update for Android 4.2 on my Galaxy Nexus. This really ruined my day. Basically the long and short of it is that the battery took a nose dive after installing this update. Immediately I went from a full day of battery life (only charging it while I slept) to approx 3 hours of battery life. As soon as that happened I changed my email checking settings to manual, so that it would not constantly poll, I also disabled wifi, bluetooth etc. However there was a little gem of information in the Battery Usage reports. Basically it was reporting that ~85% of my battery was being used by “Exchange Services” no longer was the screen the biggest consumer of my juice.

Linux KVM: Ubuntu 12.10 with Openvswitch

Wed, 24 Oct 2012 00:00:00 +0000

Today I am revisiting my previous post on Openvswitch on Ubuntu 12.04. Things have changed since then. Previously Openvswitch was relatively new and as such the userland tools (with libvirt being the one I use) didn’t support it yet, so while you could have used Openvswitch by executing the kvm processes for each VM manually. Since this was a predictable problem the Openvswitch guys created a brcompat module, which would take the place of the bridge_utils functionality and expose Openvswitch bridges as legacy bridges. This allowed the existing userland which already had the legacy bridge functionality builtin to continue to operate the same way. Anyways as of libvirt 0.9.11 the native Openvswitch functionality is present and supported. As such I am updating my documentation to include this alternate and preferred method of operation.

Windows Server 2012: NIC Teaming

Mon, 01 Oct 2012 00:00:00 +0000

I don’t tend to use a lot of Windows in recent years, but if you look at my past articles I used to do quite a bit around Hyper-V, DFS, Exchange, and various other Windows Services. One thing that was always frustrating is the inconsistent experience around NIC teaming. Basically Teaming allows you to gain throughput and resiliency by having multiple physical NICs listen over the same logical NIC. The reason of this inconsistency was that teaming was always provided by the hardware/driver manufacturer so features and configurations would be implemented separately.

Windows Server 2012: Storage Pooling

Mon, 01 Oct 2012 00:00:00 +0000

Windows Server 2012 includes a new storage model. It follows in the footsteps of ZFS and btrfs, in that it uses a pooled disk model, and attempts to combine all of the functionality of RAID, LVM, and the file system. Now this looks like a 1.0 from the perspective of integration, however I am quite impressed by the implementation from the usability perspective. While it is not as polished and integrated the flow is consistent and when using the GUI it is very simple to actually configure your storage. That said this feels like multiple layers.

Linux-LVM: Resize Partition to Grow LVM Volume Group

Mon, 20 Aug 2012 00:00:00 +0000

Logical Volume Manager makes the dynamic expansion of file systems dead stupid simple. However there is a weakness, if you are using a partitioned file system as your Physical Volume (PV) then you will end up needing to expand the file system if you ever need to grow the actual storage. This can be avoided by using the actual physical device /dev/sda as the PV, however if this is the same volume where your /boot partition lives then you will have no choice but to partition it out since /boot cannot reside on a Logical Volume (LV). The other way you can avoid this is by simply creating a second partition on the same disk and adding it as a new PV to the existing Volume Group (VG). This is preferable as there is far less room for error than in this procedure, but I personally find that method a little more confusing to inherit then a straight forward single partition made an LV.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part Two

Tue, 07 Aug 2012 00:00:00 +0000

In our previous article we went through the process of preparing a Solaris 11 for SPARC machine as a Logical Domain hypervisor. Now that we have laid out our foundation we are ready to create some Logical Domains.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Create the Logical Domain

bash

ldm add-domain t4-g1

Configure Devices

We will need to add devices to our newly created domain. Here we will add virtual CPUs, memory, and a network device.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Mon, 06 Aug 2012 00:00:00 +0000

Solaris 11 for SPARC, as well as previous versions of Solaris, include a neat little technology called Logical Domains (which has been rebranded as Oracle VM for SPARC). Logical Domains are implemented in the SPARC processor.

This series is going to focus on the basic administration of a Logical Domains environment using the basic command line tools of the Solaris 11 Operating System. In Part One we will go over what it takes to configure a Solaris 11 installation to be a Logical Domains hypervisor. In Part Two we will document defining and configuring Logical Domains. In Part Three we will discuss connecting to the Logical Domain in order to perform an installation of the operating system.

Bash: Automatically Mount File Systems on Volume Group if Present

Thu, 02 Aug 2012 00:00:00 +0000

Update 09/16/2015 – I have published an updated article for this here: http://blog.allanglesit.com/2012/08/bash-automatically-mount-file-systems-on-volume-group-if-present/

In my laptop I have a solid state disk, and frankly I am way past addicted to solid state disks, but what you get in performance you lose in capacity. As such I have had to be creative with how I can have the capacity and the performance that I need. Lately the solution has been to have a spinning disk in the slot that would normally be occupied by my CD/DVD drive. However the problem with this, revealed itself to be that my Fedora install would actually fail to boot when it tried and failed to mount the file systems if this device was not present. So my first “solution” was to set noauto in the /etc/fstab for these secondary file systems. Which presented a new problem… I would forget to manually mount these devices before launching Virtualbox to start my VMs. So I decided that I would need to have an auto-mount script which would check for the volume group to be present and then if it was mount the file systems contained within the /etc/fstab which live on the volume group.

Solaris 11 on SPARC: What the Piss is the OK Prompt

Wed, 01 Aug 2012 00:00:00 +0000

I have finally got my hands on some SPARC gear, and I am making use of it by documenting the crap out of it. So after I got the ILOM situation sorted out I noticed that the OS itself wasn’t starting up, and I was just ending up at a prompt that looks like this.

text

{0} ok

This is the OK Prompt, some systems will display ok> and this is how I will refer to it in subsequent examples. So lets dig in.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part Three

Wed, 01 Aug 2012 00:00:00 +0000

In our previous articles we built the foundation of our Logical Domains environment, which then enabled us to create a Logical Domain in part two. Now in the final article in this series we will be connecting to and performing the installation of the operating system.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Solaris Virtualization: Using Logical Domains on Solaris 11 Part Two

Before we start the domain, we need to bind the associated resources to the domain. This will also show us the port that the console is running on, so that we can connect to it.

Solaris 11: Permit Root Logins Locally and via SSH

Sun, 01 Jul 2012 00:00:00 +0000

In Solaris 11, a secondary user is created as part of the installation process, the text installation allows you to bypass this, but even if you do you will be unable to login as root to perform any configurations, even though you set the password as part of the setup. Now if you went through this in the installation, you can salvage your installation by booting into single user mode. But for those of us who just want to manage this box differently the steps are the same (minus the single user mode).

Linux-KVM: Using KVM on Oracle Linux 6

Fri, 01 Jun 2012 00:00:00 +0000

In my new role I am using Oracle Linux more and more, so as an exercise I have been replicating my builds on Oracle Linux. Here we are going to use Oracle Linux 6.2 x86_64 as a KVM hypervisor. Keep in mind that KVM is not Oracle’s preferred hypervisor, however it is supported, but if you plan on taking advantage of the reduced software licensing cost by limiting your cores for Oracle Products you will need to use Oracle VM, which is built on Xen not KVM. Read on to find out how to configure Oracle Linux as a KVM hypervisor.

LVM: Using 2TB Volumes for Volume Groups

Mon, 28 May 2012 00:00:00 +0000

Occasionally you will have the need to use physical volumes which exceed the limits of the standard DOS partitioning scheme. When you run into this a quick fdisk -l will reveal that you will need a new way to partition your disks.

Fdisk will produce output similar to this…

 text
 
 
WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

WARNING: The size of this disk is 2.3 TB (2290828181504 bytes).
DOS partition table format can not be used on drives for volumes
larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID
partition table format (GPT).

Use Parted to Create a Partition Using the Whole Disk

Parted is able to handle large volumes which are not able to be modified with fdisk. Here we are going to launch parted, and then create the partition with a size of 1 -1 which means that it will use it all until it gets to the end. You can additionally specify the number in megabytes, so 1 1024 would start at 1 and go until 1024 giving you a 1GB partition.

Solaris 11: DNS Client Configuration Using Svccfg

Wed, 23 May 2012 00:00:00 +0000

In Solaris 11, alot of configurations are being moved from configuration files into the Service Management Framework. Here we will discuss this change around the DNS client.

View Existing DNS Client Configuration

 bash
 
 
svccfg -s network/dns/client listprop config
config application
config/value_authorization astring solaris.smf.value.name-service.dns.client
config/domain astring test.local
config/nameserver net_address 10.0.0.152

Update Existing DNS Client Configuration

Here we will update our name servers. In this case we are replacing the original with two different addresses.

Solaris 11: Change Hostname Using Svccfg

Tue, 22 May 2012 00:00:00 +0000

When Solaris 11 was released more configurations were moved into the service management framework, and moved out of configuration files, one of these changes was the hostname/nodename.

View Hostname in SMF

This will read back the configuration out of the SMF.

 bash
 
 
svccfg -s system/identity:node listprop config
config application
config/nodename astring solaris11box
config/loopback astring solaris11box

Change Hostname in SMF

We have two separate settings, which need to be updated.

Oracle VM: Using Oracle VM Manager 3 from Ubuntu Linux

Wed, 16 May 2012 00:00:00 +0000

In my initial work with Oracle VM, I have run into a few fringe cases, which are not well documented, but actually have rather simple solutions. This one actually cropped up over the weekend for me. Over the weekend I decided to rebuild my company-issued laptop with Ubuntu Linux (12.04 amd64) to be specific. I did this for a number of reasons…

1) I am far more effective with Linux, due to the nature of the shell, and tools which I am familiar with (vim, dd, really take your pick they are everywhere).

Oracle VM 3.x: Reset Agent Password

Tue, 15 May 2012 00:00:00 +0000

Oracle VM, is a centrally managed hypervisor. All administrative actions must be performed from the Oracle VM Manager. In order to facilitate the initial discovery of the Hypervisor by the Management Server part of the setup requires that we not only set the password for the root user, but also the password for the ovs-agent. We then initiate a discovery process from the management server to the hypervisor and end up able to manage our hypervisor.

Installing Oracle VM Manager 3.1.1

Mon, 14 May 2012 00:00:00 +0000

In my new role at Keste, I am tasked with staying on the bleeding edge of software release cycles, so that we can be knowledgeable and authoritative about technology when we engage with customers. One of the areas where Keste leads is on Oracle’s Virtualization Offerings. So I will be posting more and more content with respect to those offerings.

Today I will be documenting the steps to install Oracle VM Manager 3.1.1. Oracle VM (OVM) is a Xen-based Virtualization Technology. An OVM Deployment consists of the OVM Server (the bare metal hypervisor) and the OVM Manager, the application stack which manages the afforementioned hypervisors (organized into Server Pools).

Bash: A Better Way to Sed

Tue, 01 May 2012 00:00:00 +0000

So I frequently find myself in a situation where I need to modify files on a fairly programmatic basis, so sed has become a friend of mine for a lot of these situations. So lets start with some basic sed…

If I have a file..

 bash
 
 
cat /etc/file.txt
blah
blah
teststring
blah
blah

And I want to replace “teststring” with “productionvalue” then I could use sed.

bash

sed -i 's/teststring/productionvalue/g' /etc/file.txt

 bash
 
 
cat /etc/file.txt
blah
blah
productionvalue
blah
blah

This is all well and good, but recently I found a better way…

Bash: Programmatically Add Entries in fstab

Tue, 01 May 2012 00:00:00 +0000

So I have found that it is rather easy to add entries into /etc/fstab while maintaining relatively legible scripts and making the changes with minimal user interaction…

METHOD ONE

This option is the simplest, we simply echo the line into the file as we want it to appear, then based on the » it is added as the last line in the file. This is fine, however this does not provide us with a way of inserting tabs, it is simply whitespace.

Solaris Virtualization: Using Zones

Tue, 01 May 2012 00:00:00 +0000

Solaris has the ability to do two types of Virtualization, if you are using SPARC hardware you can use Logical Domains, which has been re-branded as Oracle VM for SPARC. Or regardless of your architecture you can use Solaris Zones, which is really much more like Linux-VServer, OpenVZ, and Linux Containers (LXC) than a “full” hypervisor solution. Essentially you create a section of the file system on the host (in the case of ZFS a separate file system) and this is used by the “Zone” as the “root” or / and that is as far as the guest system can go. This is a very fast form of virtualization because it doesn’t really virtualize anything.

Upgrade Solaris 11 Express to Solaris 11

Tue, 01 May 2012 00:00:00 +0000

Check Current Version

The important thing to notice here is snv_151a, which means that it is Solaris 11 Express.

bash

uname -a
SunOS storage01 5.11 snv_151a i86pc i386 i86pc Solaris

 bash
 
 
cat /etc/release
Oracle Solaris 11 Express snv_151a X86
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
Assembled 04 November 2010

Display the Current IPS Repositories

This will show you the repositories configured on your system.

 bash
 
 
pkg publisher
PUBLISHER TYPE STATUS URI
solaris origin online http://pkg.oracle.com/solaris/release/

Configure the IPS Repository

Bash: Duplicate Logical Volume Configuration for a Volume Group

Mon, 30 Apr 2012 00:00:00 +0000

I recently ran into a situation where I needed to create a large number of Logical Volumes for a server replacement situation. I could have done some manual data input, but why not take the opportunity to quickly write a script to do the hard work for me?

For my situation this was a server that pulls data from other servers in order to optimize getting that data to tape. So we had 2 VGs, one for the system and one for the data we wanted to move. We planned on simply performing a rsync to move the data itself, but the volumes had to be pre-staged.

ESX 5: Command Line VM Manipulation

Tue, 24 Apr 2012 00:00:00 +0000

I haven’t been working with ESX for a bit, but while I was I was mostly interacting with it via the command line. Here is a list I had compiled while I was going through that exercise, and since I while be starting a job next week where I will no longer be interacting with much ESX I will just publish this one as-is. It is by no means complete and frankly it is not even that helpful. Notice that the commands which reference a specific virtual machine have a number, this is the VM ID for the VM you’d like to interact with.

Preparing Minimal Images with Debootstrap

Tue, 27 Mar 2012 00:00:00 +0000

Debootstrap gives us a simple and consistent way to cleanly setup machines by installing packages directly into a file system directly from the repos. These have the flexibility of being used across many different kind of machines.

Create Directory Structure

Here we are just creating the directories into which we will be installing the operating systems into.

 bash
 
 
mkdir -p /mnt/build/amd64 /mnt/build/i386
cd /mnt/build/amd64; mkdir ubuntu_10.04 ubuntu_10.10 ubuntu_11.04 ubuntu_11.10 debian_5 debian_6
cd /mnt/build/i386; mkdir ubuntu_10.04 ubuntu_10.10 ubuntu_11.04 ubuntu_11.10 debian_5 debian_6

Install Required Software

bash

apt-get install debootstrap

Install Operating Systems into the Folders

Ubuntu Lucid

Linux-KVM: Ubuntu 12.04 with Openvswitch

Mon, 26 Mar 2012 00:00:00 +0000

UPDATE: October 29, 2012

This article doesn’t work on Ubuntu 12.10, to use Ubuntu 12.10 please refer to my new article here.

Openvswitch is a better way of managing your virtual networking stacks for KVM and Xen. This can be used instead of the bridge-utils package, and has the ability to use VLANs, LACP, QoS, sFlow, and others. In this article we will be using Ubuntu 12.04 (beta1) to configure Openvswitch to hand out networking interfaces to KVM guests.

Adventures in ZFS: NexentaStor with Dell Poweredge R515 Onboard Broadcom NIC

Thu, 01 Mar 2012 00:00:00 +0000

So recently I dealt with a rather aggravating problem. Basically I had a couple of Poweredge R515’s to be built with NexentaStor Enterprise, which would not get a valid network configuration.

The Problem

During the initial setup via the console… I selected bnx0 as the primary and set it to use DHCP, after a bit of chewing I received the error below…

text

IOError: can not safely set interface bnx0 via DHCP.

After the wizard I went into the shell to look at the configuration.

Equallogic: Remove Volume Snapshot Reserve

Thu, 01 Mar 2012 00:00:00 +0000

So if you purchased a Equallogic and put it into production one of the first things that you notice is that your xTB didn’t go as far as you originally planned, this is because Equallogic by default comes with a 100% snapshot reserve on all volumes that you create. This means that if you allocate a 1TB LUN, then the system provisions 1TB of LUN space and an additional 1TB of snapshot reserve for this LUN. This is neither good nor bad. However this is not always necessary. If you plan on making little use of snapshots you may want to decrease this amount of reserve or in my case remove it entirely (as we don’t plan on using it at all).

ESX 5: Link Aggregation with Netgear GSM7352S

Thu, 01 Mar 2012 00:00:00 +0000

As one of our final validation points in an ongoing project has been enabling ESX to use a LAG on both the Management and VM Networks. The last time we did this we were unable to get it to work, and it turns out the solution was actually very simple. Basically the core of the issue was that we were attempting to use Dynamic LACP which ESX does not support, instead you are to use Static LACP.

Gnome3: Disable Hot Corners

Thu, 01 Mar 2012 00:00:00 +0000

I sincerely hate Hot Corners in Gnome3. This article will walk you through disabling them, because you probably hate them as much as I do (you’re here right)…

So firstly what are hot corners, and why do I hate them? Hot corners is the upper left hand corner of your screen (0,0) the idea is that if you move your mouse into the upper left hand corner then you must want the dash to come up. However it frustrates me, because sometimes when I am typing or cussing or both, I will inadvertently bump my mouse and then I end up in the dash.

KVM Guests: Windows NT… What?

Thu, 01 Mar 2012 00:00:00 +0000

OK so you have been kicking around an old NT machine for years, neigh, a decade. So what do you do with it? Realistically the machine has already sustained numerous failures, and it won’t make it through another one. We have all fought this battle. I personally fought this battle many years ago and frankly P2V is never clean when coming from NT. But if you could upgrade then you would have already.

Microsoft Online Support: The Verbal Blue Screen of Death

Thu, 01 Mar 2012 00:00:00 +0000

So today we had the need to call Microsoft Online Support, we were definitely suprised, to experience what I have coined as the “Verbal Blue Screen of Death”. So apparently, if you are in need of Microsoft Online Support then tough cookies. This was as of 2PM on March 20, 2012. Below find the link to the MP3.

verbal-bsod

Adventures in ZFS: Configuring iSCSI Targets

Mon, 27 Feb 2012 00:00:00 +0000

The most critical thing about any storage implementation is flexibility. This means that you need to be able to adapt your solution to changing environmental considerations. The nice thing about a ZFS-based solution is that all of the building blocks are available. If you want to use it as a NAS device, you can use CIFS, NFS. If you want to provision Fibre Channel that is available too. Even some of the more sought after SAN features are included, replication, deduplication, thin-provisioning, and many more. Now in todays article we are going to discuss how you can go about setting up iSCSI targets. I personally am not a fan of iSCSI (I have been bitten by the hard knee too many times), however you really can’t argue with the flexibility that is provided by being able to quickly provision storage on already existing or readily available hardware. The best thing about this process is that it is really similar to provisioning fibre channel targets, so if you are familiar with this process (available here) then you will be able to quickly adapt to the iSCSI variation.

Securing SSH with Publicly Accessible Servers

Thu, 16 Feb 2012 00:00:00 +0000

Just about every IT environment has some sort of remotely managed environment which requires that they have SSH open to the Internet. Perhaps this is a VPS, dedicated server, or colocation. Regardless of your reason, the fact is that there is just some times where you need to have SSH open to the internet.

However regardless of if it is necessary doesn’t mean that you should just do it… One look at your auth.log will reveal that.

vCenter: VPXD Crash

Wed, 15 Feb 2012 00:00:00 +0000

Recently I was trying to resolve an issue on a small vCenter 5.0.0 deployment. Basically the primary symptom of the issue was that the service the vmware-vxpd service was crashing and needing to be restarted frequently.

The first thing I did was examine the log files, while the service was still in a failed state.

bash

cat /var/log/vmware/vpx/vpxd.log

I won’t bore you with an entire excerpt, however I will put some of the more helpful pieces of data from the log file here, perhaps it will help someone identify this article if they are having a similar issue but not really gaining any traction.

KVM Guests: Graceful Shutdown

Mon, 06 Feb 2012 00:00:00 +0000

So a fairly trivial (but critical) aspect of using KVM is of course performing graceful shutdowns of your domains, without having to reach inside of the guest to perform the shutdown. Now when it comes to turning off your guests you have two ways of proceeding (with virsh)…

bash

virsh destroy vmname

Which is a HARD power off of your guest. This is the same as pulling the power cord from the wall, nothing is shutdown, but the noise stops just the same. This method can result in data corruption if writes are interrupted.

The Changing Face of Enterprise Support

Fri, 03 Feb 2012 00:00:00 +0000

Now let me preface this post by saying…

I am writing this article from my professional experiences, inside of a software company which provided enterprise support for its products, as well as from the perspective of the Enterprise Customer. I am intentionally discarding all experiences with support as a home user, and even as a small business owner. This is specific to Enterprise Support, consumer support plays by different rules.

Adventures in ZFS: Storage Sizing

Wed, 01 Feb 2012 00:00:00 +0000

One of the most complex parts of storage in general and ZFS in particular is correctly assessing the amount and types of storage that you will need to meet your requirements. You can of course just purchase what you can afford and hope for the best. But after reading this article you can relatively easily determine approximately how much storage you should plan for based on a few factors from your existing environment, this article will primarily focus on ZFS, however alot of this is true in other storage platforms as well…

Dell PowerConnect 6224: Stacking Module Dual Master

Wed, 01 Feb 2012 00:00:00 +0000

We use a couple of Dell PowerConnect 6224 switches in our storage network, however when they were first deployed we ran into an issue with the stacking modules. Basically when you brought up the switches they would both see themselves as master, and would not “stack” correctly. Turns out the solution was rather easy but not well documented. Also as a side note, I have a friend with a couple of PowerConnect 6248 switches where he had the same issue. So it appears this issue spans the whole family at least.

iSCSI Initiator: OpenIndiana

Wed, 01 Feb 2012 00:00:00 +0000

Alot of my time is spent handling storage from the array side, however recently I had the need to test both sides of the process in order to perform some exclusionary tests against a fully adaptable environment, without impacting our production environments, so I ended up using OpenIndiana 151a on both the target and initator side of the connection. This is how you handle the initiator side of the connection.

Installing OpenIndiana in a Xen DomU

Mon, 16 Jan 2012 00:00:00 +0000

So I am on the ground in Dallas and trying to get my head around some of the technologies that we use in my new environment. One of those being the Xen kernel. I have never actually used Xen before, though I have used xenserver a bit. Anyways so my first foray into the Xen world is trying to get OpenIndiana (successor to OpenSolaris) running on it, and see what sort of performance issues it might have. We are hoping to be able to leverage its CIFS and vscan functionality to provide some security for a secured network file server.

Adventures in ZFS: Crucial M4 Firmware SMART Issue

Sun, 01 Jan 2012 00:00:00 +0000

Let me preface this article by stating that this issue is NOT a ZFS issue or a Solaris issue. Previously I documented a issue in which the Crucial M4 SSD would “hesitate” causing ZFS to see it as faulted. This was promptly fixed in v0002 of their firmware. That issue is documented here. This issue is a bit more aggravating and has not been fully resolved.

The issue is that after a drive has been operating for 5184 hours it will cease to respond to the system. After a reboot the drive will work again, although the problem will repeat itself after an hour. This will continue to happen until a fix is implemented in the firmware.

Mass Extraction and Creation of Tarballs

Sun, 01 Jan 2012 00:00:00 +0000

I have been working on creating a bunch of debootstrap images lately, and I ended up doing a whole bunch of work creating tarballs and did it wrong. So I took the time to sort out an easy way to fix them all with two commands.

The Core of My Problem

So what started this is I forgot to “stand” in the directory I was archiving, as such all of my images ended up having a top level directory named lenny32 (insert appropriate distribution and architecture here) which isn’t very helpful when you are extracting it to the root of a file system.

Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes Redux

Mon, 07 Nov 2011 00:00:00 +0000

This really is an update to my previous article “Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes“, but the process is significantly different and thus requires more explaining than a simple inline update would do it justice. Now the big flaw with my first article was that by using dd on the hypervisor level, we are opening ourselves up to unintentional data loss due to incompetence or even a very competent slip of the finger.

SSH Hop Through Multiple Hosts

Tue, 01 Nov 2011 00:00:00 +0000

I frequently need to ssh through multiple hosts to get to a destination. Frankly this is not really a problem, but there is a simpler way, and I like simpler. This is a really straight forward article, not a lot of explanation is really necessary. Sometimes I need to use this when I am connecting through hosts that are not fully routed, or perhaps to get to a secure network – where you must traverse a firewall.

Using Curl to Interact with Resource Records in Rackspace Cloud DNS

Tue, 01 Nov 2011 00:00:00 +0000

In this article we will be using curl to interact with Rackspace’s Cloud DNS to manipulate resource records in zones which already exist. For details on how to use curl against Rackspace’s Cloud DNS API please see part one in this series “Using Curl to Access Rackspace Cloud DNS” in part two I documented the creation of DNS zones “Using Curl to Interact with Zones in Rackspace Cloud DNS” please refer to these for the lead up to this article.

Using Curl to Interact with Zones in Rackspace Cloud DNS

Tue, 01 Nov 2011 00:00:00 +0000

In my previous post “Using Curl to Access Rackspace Cloud DNS” we documented the basics of authenticating with the Rackspace Cloud for use with the Beta DNS service they offer, as well as doing basic queries against this service. Today we are going to focus on creating our first domain, which no doubt will offer challenges for us, since it will require the uploading of information to the Cloud service using the API. If you are not familiar with how to authenticate with the Rackspace Cloud please review my previous post.

Solaris 11: User Renames Not Simple Anymore

Sat, 01 Oct 2011 00:00:00 +0000

As part of the Solaris 11 Express Installation, you are required to create a named user, which is then the account you can log in as, no root for you. Anyways the problem here is that we had some machines which we subsequently wanted to change the name. Normally this would be simply handled by simple edits to the /etc/passwd and /etc/shadow. However when we made this change we found that boy were we wrong.

Using Curl to Access Rackspace Cloud DNS

Sat, 01 Oct 2011 00:00:00 +0000

Lately I have been looking at some cloud options for my more critical services, in this I have investigated Zerigo, Route 53, and Rackspace Cloud DNS. Currently I have been focused on Rackspace DNS, since it is still free at this point, and frankly I’d rather not spend money on anything until I understand exactly how it works. I have no experience with using “api” interactions and as such it has been a real struggle for me to get my head around sending the correct data to the infrastructure and getting the correct responses.

Adventures in ZFS: Time Slider is not Expiring Snapshots on Imported Zpools

Tue, 06 Sep 2011 00:00:00 +0000

A few days ago someone contacted me with an interesting issue with regards to the ZFS automatic snapshots, facilitatedby the time slider service.

The Situation

OpenSolaris machine with zpool exported. Time slider was working prior to export.
Solaris 11 Express fresh install with zpool imported.

The Symptoms

Basically the core of the issue was that the zpool was from an OpenSolaris machine, in lieu of doing an upgrade from OpenSolaris to Solaris 11 Express, the zpool was exported and imported into a machine which was already running Solaris 11 Express. Upon the import new snapshots were being created by the time slider service however old ones were not being expired. You can still manually delete snapshots.

Adventures in ZFS: Configuring Fibre Channel Targets

Wed, 24 Aug 2011 00:00:00 +0000

In our environment ZFS has become a critical component of our Storage Infrastructure. We have been able to provision Fibre Channel storage from our ZFS file systems which give us the benefit of data integrity, deduplication, performance through use of the ARC, L2ARC, and ZIL as needed. Additionally the real benefit to ZFS is our storage can run on commodity hardware which reduces the TCO significantly over the major Storage vendors. Now administering ZFS is really quite simple, however getting Fibre Channel can be a bit more work. However after reading this you ought to be able to handle this with your eyes closed, well maybe not with your eyes closed. But you will be much better prepared.

Linux-KVM Management: Live Migration Without Shared Storage

Tue, 16 Aug 2011 00:00:00 +0000

Everyday I learn something new about Linux-KVM, this is why I love this platform. Today we are going to talk about Live Migration. The kicker? Shared storage not required. Now please keep in mind that just because shared storage is not required, doesn’t mean there is not a benefit to it. Shared storage will accelerate this process, since the data will not have to move. However virtualization is all about flexibility and so is Linux for that matter, so why shouldn’t we be able to make the choice? That said this process is not documented by the libvirt team, as such it sounds like an “undocumented” feature. Please use this at your own risk, and for heaven’s sake please test and understand thoroughly before using this in production.

Adventures in ZFS: Faulted Cache Devices

Mon, 15 Aug 2011 00:00:00 +0000

We recently had an issue with Crucial M4 Solid State Disks when using them with ZFS on Solaris 11 Express (snv_151a). Basically the disks were showing a whole bunch of write errors and had been “FAULTED” by ZFS. Now to make this problem even worse when we tried to zfs clear them it locked up my SSH session, as well as subsequent sessions, it would allow me to initiate subsequent sessions (and authenticate), however would not deliver me to the prompt. Additionally when I went onto the console if I ran a “zfs list, zpool list, or zpool status” it would lock up the shell every time. When I say lock up, I mean lock up, no CTRL + C or Z or anything. So since these devices were cache devices only this wasn’t the end of the world for us.

Bash: Using Temporary SSH Keys Within A Script

Wed, 10 Aug 2011 00:00:00 +0000

Every once and a while you will have the need to make multiple ssh calls to remote systems within a bash script. Normally you would simply have the user enter their password multiple times, or have a requirement for SSH keys to be configured prior to running the keys. This is widely accepted as the way to do it, however if that were the best way to do it, then I would not be writing this article. Now the better way to do it would be to create ssh keys, and upload them as authorized keys to the remote host(s) then upon completion of the work we can then clean up the changes that we made.

Bash: Using pv to Display Progress of dd

Tue, 09 Aug 2011 00:00:00 +0000

One of the biggest weaknesses of dd is that it has no way to display progress on its actions. You can send a signal to the process which will pause, display statistics, and resume the process however this takes up alot of your terminal screen if you are doing any sort of long running copy. Enter pv. Pv allows us to monitor the progress of data through a pipe.

Create A 1GB Test File

This is the file that we will use to peform the following tests.

Linux-KVM Management: Offline Migration

Mon, 08 Aug 2011 00:00:00 +0000

When you compare Linux-KVM to Hyper-V or VMWare your initial results will indicate that Linux-KVM is lacking when it comes to management tools, and basic functionality. You would be correct, however you would also be incorrect. You see with Linux-KVM we can leverage the underlying power of the Linux userland, and with this frankly all things are possible. Here is one of the basic bits of management functionality which can be attained with a little bit of bash scripting knowledge. I started by writing a VM backup script, then a VM export script, and of course a VM import script. Eventually I ended up with a full-blown end-to-end VM migration script.

Adventures in ZFS: Timestamps Ruined My Rsync

Mon, 01 Aug 2011 00:00:00 +0000

So let me start out by saying… This isn’t so much of a problem with ZFS, as it is just a problem with the tools available in Solaris (not to mention whatever device actually screwed up the timestamp in the first place). But so here is the story. We have found that migrating our lower level file servers to Solaris on top of deduped or compressed ZFS is really fantastic for our environment. So I have been working on migrating more and more data to this pool of storage in order to free up some of our more expensive storage for other data.

Traffic Limiting with PfSense 2.0 RC3

Mon, 01 Aug 2011 00:00:00 +0000

PfSense is a FreeBSD-based firewall distribution, which is extremely flexible for both businesses and individuals. Additionally this platform can be installed on small ALIX hardware with Flash Memory as the storage all the way up to a full server deployment. To give you an idea of speed, the small ALIX box (I have a few) is easily able to push 20Mbps. I have a slightly larger box with spinning disks and an Atom processor, which is able to push 60Mbps (which is the maximum on my WAN). In addition to this though with the larger box I am able to deploy other packages on my PfSense box, such as Squid and many many more.

Identify Package Origin of Binary on Apt Systems

Fri, 01 Jul 2011 00:00:00 +0000

There will come a time when you need to install a package on a machine in order to take advantage of a utility contained within. In my case I know that I need kvm-img (which is also called qemu-img), and I assumed that it was either in the libvirt-bin package or the qemu-kvm package. It turned out for me that it was in qemu-kvm which is not what I was hoping. But I digress, here are two ways to get this done for anything.

Solaris 11: Mount SMB Share Locally

Tue, 31 May 2011 00:00:00 +0000

Currently I have been working on a method of migrating some of our data from Windows file servers to Solaris ZFS CIFS servers, of course trying to retain as much feature parity as possible. Permissions are an issue, however I have worked through most of those issues (look for this article in the future), now it is time to migrate data from Windows onto ZFS. This first crack will be about 1TB of data. Enter rsync. The “easiest” way to do this would be to use a Linux box in the middle to mount both the Solaris CIFS share and with Windows CIFS share and perform the rsync from there. A far more efficient way would be to mount the Windows CIFS share on the Solaris box or vice versa and perform the rsync between the two boxes. This article will document how to mount a Windows CIFS file share onto the local file system of the Solaris 11 Express box.

Linux-KVM: VLAN Tagging for Guest Connectivity

Sun, 01 May 2011 00:00:00 +0000

Today we will be discussing the process of configuring VLANs for the KVM Guests to use to gain access to a wider variety of networks from a single host. This is extremely helpful if you are able to take advantage of 10Gbps connections, as you can eliminate multiple 1Gbps connections for a single 10Gbps and still get a net gain in bandwidth. If you do not use 10Gbps yet, but still take advantage of VLANs then this will still help you reduce the cabling for your hosts.

So You Want to Learn ZFS Part One

Mon, 04 Apr 2011 00:00:00 +0000

Enterprise Storage is currently broken. IT departments big and small spend way too much money on completely ineffective storage solutions which only partially fulfil the companies storage need. All the while having to justify massive expenditures to the business which do not meet the need. Essentially IT departments everywhere have become storage sales reps, sure we work with vendors who provide us with quotes, but when the rubber meets the road it is the IT department which is selling the storage to the business. If any of you have ever been in this position as I have and literally felt sick to your stomach because you knew it was all wrong, then this article is for you. This article does not provide an Enterprise Storage cure-all. What it does provide is an interesting look at a solution which really revolutionises the entire Enterprise Storage model. More importantly it allows you the flexibility, features, and densities needed to make it far superior to any modern day Enterprise Storage solution. Also it is pretty darn cheap.

Active Directory: Identify Stale Computer Accounts

Fri, 01 Apr 2011 00:00:00 +0000

Using the Active Directory Modules included with Windows 7 + Windows 2008 R2 we can use powershell to identify computer accounts which are “stale” and have not changed their passwords in a specified amount of time.

Define the Windows for “Stale”

Example will check for 365 days old.

 PowerShell
 
$LastSetDate = [DateTime]::Now - [TimeSpan]::Parse("365")

Count the Number of PCs which are Stale

 PowerShell
 
(Get-AdComputer -Filter {PasswordLastSet -le $LastSetDate} -Properties PasswordLastSet -ResultSetSize $null | ft SamAccountName,PasswordLastSet).count

Output a List of the Computer Accounts

Exchange Management: Locate Email Addresses

Fri, 01 Apr 2011 00:00:00 +0000

If you have a large Exchange environment with a large amount of users, contacts, distribution lists, and public folders you can end up “losing” email addresses. It can take a lot of time to perform the different types of searches required to locate an email address. Or you can use powershell and locate it quickly.

 PowerShell
 
Get-Recipient -ResultSize Unlimited | Where {$_.EmailAddresses -Match "bob@allanglesit.com"} | Select Name,EmailAddresses,RecipientType

So You Want to Learn ZFS Part Three

Fri, 01 Apr 2011 00:00:00 +0000

This is the final part in the “So You Want to Learn ZFS” series. In Part One we discussed the reasons why you should be investigating ZFS if you are not already. Part Two highlighted some of the differences from a usability perspective of Solaris versus a more traditional Linux system. In Part Three we will be discussing what things you need to look for in a test ZFS system, as well as some hints on configuration of the basic system. I am assuming that you are going to use Solaris 11 Express.

So You Want to Learn ZFS Part Two

Fri, 01 Apr 2011 00:00:00 +0000

In my previous article we discussed the main reasons that you ought to be investigating ZFS as a storage platform for your organization. In this article we will be documenting some of the major pain points that people have when giving Solaris a shot. One common theme that you will notice throughout my writings about Solaris is that I am a firm believer in the over-engineering of Solaris by its engineers, I think that just about everything that feels wrong in Solaris is actually the “most correct” way however in a lot of cases it is simply not the “best” way.

Windows Client Management: Remotely Enable DHCP

Fri, 01 Apr 2011 00:00:00 +0000

Using WMI and Powershell we can remotely configure a client machine to use DHCP as long as you (1) have administrator rights on the remote machine (2) want all interfaces to be DHCP (3) know the computer name or IP of the remote machine (4) have Powershell installed on your machine, remote machine is not required.

Check Network Configuration

 PowerShell
 
Get-WmiObject -ComputerName RemoteComputerNameOrIP -Class Win32_NetworkAdapterConfiguration

Check Network Configuration (only where IP is enabled)

ZFS: Clone Management

Fri, 01 Apr 2011 00:00:00 +0000

In our previous article “ZFS: Snapshot Management” we learned about the many methods of using snapshots to give you a shorter path to recovery as well as reduce the time needed to revert changes that need to be reversed. Today we will discuss clones, which are essentially a copy-on-write duplicate of an existing snapshot, which unlike snapshots can be written to, can have snapshots taken against them, and can be promoted into a file system (which would then revert to the original file system as the clone).

ZFS: Delegate ZFS Permissions

Fri, 01 Apr 2011 00:00:00 +0000

It is widely accepted as best practice to not use your root account in general. In Solaris they like some Linux distributions have gone so far as to prevent root access to SSH. Now we could go the route of permitting route logins to SSH however this would not be ideal from a security perspective. Instead we will be going about it a the “proper way” and allowing the delegation needed to perform actions via a named user.

ZFS: Snapshot Management

Fri, 01 Apr 2011 00:00:00 +0000

So snapshots are basically a point in time view of everything which exists on a file system at a given point in time. They allow you to build complex data protection schemes which can allow you to give more user-directed restore capabilities, as well as simply give you a good way to roll back complex changes without the time and hassle of restoring large amounts of data. Now before we go into how to use snapshots, it is probably very important to mention what snapshots are NOT.

Solaris 11: Join AD Domain for Samba

Wed, 30 Mar 2011 00:00:00 +0000

Configure Kerberos

Make a backup of our original, just in case.

bash

cp /etc/krb5/krb5.conf /etc/krb5/krb5.original.conf

These sections already exist in the file so you will want to replace them with the correct values for your environment.

 bash
 
 
vi /etc/krb5/krb5.conf

[libdefaults]
default_realm = ALLANGLESIT.COM

[realms]
allanglesit.com = {
kdc = dc.allanglesit.com
admin_server = dc.allanglesit.com
kpasswd_server = dc.allanglesit.com
kpasswd_protocol = SET_CHANGE
}

[domain_realm]
.allanglesit.com = ALLANGLESIT.COM

Quick recycle of the services or a start if they aren’t running.

Solaris 11: Network Configuration Advanced

Tue, 29 Mar 2011 00:00:00 +0000

In my previous post we went over the basics of configuring a network interface in Solaris 11, without using the Network Auto Magic (NWAM) which is enabled by default. In this article we will go over some of the more advanced features which can be leveraged including VLANs, aggregation groups, and jumbo frames.

Configure VLAN Tagging

Create a VLAN, specify the VLAN id with the “-v”, then specify the interface with the “-l”, and finally create a name for the tagged interface (in this case user0 since this particular system is the user VLAN, if you have more descriptive names of your VLANs you can use them here).

Solaris 11: Network Configuration Basics

Mon, 28 Mar 2011 00:00:00 +0000

I have been doing research for my next big series “So You Want to Learn ZFS.” This series is basically going to be a multi-part series of How-To’s which hopefully will give you the ability to build a file server (or even a SAN) based on ZFS if you so choose. However there are a few things that I failed to take into account.

Solaris 11 is so different from Solaris 10.
Solaris 11 is so different from EVERYTHING else.

I figured that I’d be able to kind of gloss over the high points of how to get your system up and running and just dive right into the fun ZFS stuff. So before we get into the good stuff there are some basics that we will need to go over first. Today we will cover basic networking.

Linux-KVM: Migrating Hyper-V VHD Images to KVM

Wed, 23 Mar 2011 00:00:00 +0000

I have always been a big fan of Hyper-V, and was actually a Virtualization MVP 2 years in a row. However recently I haven’t been able to get past the really poor support for Linux on the Hyper-V platform. We run a large number of hypervisors in our international sales offices (15+) and have a need to run both Linux and Windows on them. This has necessitated our switch to Linux-KVM as their Windows support is far better than Hyper-V’s Linux support. As such with an upcoming trip planned to our European offices I have been planning exactly how our migration will look, of course the first iteration of this plan consisted of me deploying replacement VMs on the KVM stack then migrating to the VMs individually. Though this plan was lengthy and cumbersome it would work.

KVM Guests: Install VirtIO Drivers for Windows Guests

Tue, 22 Mar 2011 00:00:00 +0000

Windows guests on the KVM hypervisors can get a very large kick in the pants when it comes to performance if you install the drivers necessary to leverage the VirtIO bus. So first here are the necessary download links so that you can download the drivers.

http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/bin/virtio-win-1.1.16.iso

http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/

Inside of the ISO we will find four directories Balloon, NetKVM, vioserial, and viostor. We are only concerned with viostor and NetKVM, for the storage and networking drivers. Vioserial is a serial interface which essentially lifts the single serial device limitation within KVM, and Balloon is a balloon memory driver I personally do not find this necessary due to the efficacy of the Kernel Samepage Merging (KSM) which is essentially memory deduplication.

Linux-KVM: Managing KVM Guests Using Virt-Manager on Windows

Mon, 21 Mar 2011 00:00:00 +0000

When I first started using KVM, I was quite disappointed with the graphical management abilities of KVM. Basically if you want to manage via the command line KVM is a fantasical dream world where anything is possible and it is all “musical sunshine” and “double rainbows” but if you want the ability to manage graphically it was a world which was very one sided in that if you wanted the ability to graphically manage your VMs then you had to use Linux. Or so I thought…

Linux-KVM: Managing Disk Images

Wed, 16 Mar 2011 00:00:00 +0000

Disk images have long been the traditional form of storage for virtualized environments, they are essentially containers in the form of a file on the host’s file system. These files can be either fully allocated or sparsely allocated at time of disk creation, a fully allocated 20GB disk image will take up 20GB of storage on the host file system, a sparsely allocated 20GB disk image will only take up as much storage on the host file system as has actually been written inside of the disk, so if you have a newly created sparse file the file will be just a few KB in size.

KVM Guests: Manipulating Libvirt XML for Guest Creation

Tue, 15 Mar 2011 00:00:00 +0000

Today we are going to discuss using the underlying XML which libvirt uses to define a VM to either create VMs or to make simple changes. This method has the advantage of more control over what you are creating, although it is more manual. Now first off, when you use virt-install, virt-manager, or another tool which uses libvirt under the covers an XML file is created for every VM that you define, libvirt stores these in /etc/libvirt/qemu/ and names them vmname.xml.

KVM Guests: Using Virt-Install to Install VMs from a CD or ISO Image

Mon, 14 Mar 2011 00:00:00 +0000

In my previous article KVM Guests: Using Virt-Install to Import an Existing Disk Image we discussed how to use virt-install to import an existing disk image, which already has an OS installed into it. Additionally in KVM Guests: Using Virt-Install to Install Debian and Ubuntu Guests I documented how to initiate an install directly off of the apt mirror of your choice for Debian and Ubuntu Guests using virt-install. In this article we will use virt-install to create a guest and begin the installation using a CD or ISO image for installation media.

Linux-KVM: Taking Advantage of Memory Deduplication

Thu, 10 Mar 2011 00:00:00 +0000

Kernel Samepage Merging (KSM) is essentially active memory deduplication. What does this means for our KVM guests? Basically the more similar guests we have running, the more of a memory footprint they share, which means less memory they actually hold. These free pages can then be reutilized by the host to provide improved i/o through some of the caching that KVM does or by an increased capacity for other guests.

Linux-KVM: Converting Raw Disk Images to LVM Logical Volumes

Wed, 09 Mar 2011 00:00:00 +0000

Linux Logical Volume Manager provides a better method for managing your disks in the home and even more so in the datacenter. I briefly outlined some of the benefits afforded us by LVM2 in a previous article Linux LVM2: Flexible Local Storage Management. In this article I will provide you with some very simple steps and scenarios to migrate your existing KVM VMs to LVM2 Logical Volumes.

In this article I am going to make an assumption that your disk images are raw disk images and that you are not using qcow2, vhd, vdi, vmdk or any other disk image format. Additionally this process should work for a physical or RAID device as well, though I have not verified that.

KVM Guests: Using Virt-Install to Import an Existing Disk Image

Tue, 08 Mar 2011 00:00:00 +0000

In my previous article KVM Guests: Using Virt-Install to Install Debian and Ubuntu Guests we identified how we can use the virt-install command to create a KVM VM and instead of using an iso or physical cd for the install media we can instead use a boot image from an apt mirror via http. Now obviously this does not harness the true power of the virt-install command. Today we will discuss how we can take an existing disk image and import it as a VM with the appropriate devices, this would most commonly be done with sysprepped Windows images, to speed up deployment. Though these instructions will easily work with any OS.

KVM Guests: Using Virt-Install to Install Debian and Ubuntu Guests

Wed, 02 Mar 2011 00:00:00 +0000

I like to be able to spin up a VM without looking for CD media so for a while now I have been using ISO images for all of my installs directly, however I found my self scurrying around “digitally” looking for the correct iso in much the same way I used to scurry about looking for physical CDs, except I was not physically moving. So anyways the time was wasted either way, I figured it is time that I find a better (read: lazier) way to do this. So the idea is that since the apt package manager contains all the packages and files needed to install and upgrade a system, we ought to be able to find a way to initiate the install over the wire.

Linux LVM2: Flexible Local Storage Management

Tue, 01 Mar 2011 00:00:00 +0000

Linux LVM provides a different way of looking at storage. On the server side this provides much more flexibility than standard disks with partitions. It allows us to migrate from old to new storage as well as expand the available storage to a particular file system easily (assuming that there is free space available).

Basically LVM abstracts the physical storage by creating Physical Volumes (PV), and each PV is divided into Physical Extents (PE) which are a consistent size. Physical Volumes are then added to a Volume Group (VG) which essentially pools all of the PE on all member PV. Once you have a VG you can then create a Logical Volume (LV) which can be formatted and used as if it were a physical disk, the difference is that the LV can be expanded as many times as needed up until all PE have been allocated within the VG. In this event you could then simply add another PV to the VG, which would make available additional PE to be added to a LV.

Using Views with BIND 9

Thu, 24 Feb 2011 00:00:00 +0000

In almost all organizations the network infrastructure needs to be designed in such a way as to allow both internal and external name resolution authoritatively. In most organizations this has been accomplished by having separate internal and external servers. Clearly this way is functional and simpler, however it is also wasteful considering how little resources DNS actually requires. BIND 9 gives us a new method to manage these types of configurations. The idea is that you can create a view which will reference specific zones based on the network location that you are coming from.

Create a Bridged Interface in Ubuntu

Tue, 01 Feb 2011 00:00:00 +0000

Linux-KVM and Linux Containers both require a bridged interface so that it can be shared with the guests. This can also be useful if you want to share a network configuration between a wireless card and a wired card (though I will not be going into this particular configuration). Configuring a bridge is a pretty straight forward process.

 bash
 
 
apt-get install bridge-utils
cat /etc/network/interfaces
## This file describes the network interfaces available on your system
and how to activate them. For more information, see interfaces(5).
The loopback network interface
auto lo
iface lo inet loopback
The primary network interface
auto eth0
iface eth0 inet static
address 192.168.0.200
netmask 255.255.255.0
gateway 192.168.0.1
auto eth1
iface eth1 inet manual
auto br0
iface br0 inet static
address 192.168.0.201
netmask 255.255.255.0
gateway 192.168.0.1
bridge_ports eth1
bridge_stp off
bridge_maxwait 0
bridge_fd 0

As you can see above I have eth0 configured as a standard interface, while eth1 is initialized but not configured. The configuration actually lives on br0, and bridge_ports defines the member as eth1.

Using Views with BIND 9 Part 2

Tue, 01 Feb 2011 00:00:00 +0000

BIND 9 gives us the ability to run a split DNS configuration on a single server, in this article we will go over the configuration of slave servers which will transfer the zones from the master while still maintaining its views.

Now before we go any further, if you do not have a working master server with views configured you will want to review part 1 here.

Configure the Slaves

Eight Things You Need To Know About Linux on Hyper-V

Sat, 01 May 2010 00:00:00 +0000

#1 All major Linux distributions will work out-of-box on Hyper-V

By far one of the most common misconceptions is that Linux does not work on Hyper-V or that only a small number of distributions do. This could not be farther from the truth. Out-of-box most distributions will work. I personally have installed Gentoo, Debian, Ubuntu, CentOS, Fedora, RHEL, OEL, SLES, OpenSUSE and even some non Linux such as FreeBSD, OpenSolaris.

Ubuntu and Hyper-V: The Paths to Enlightenment

Sat, 01 May 2010 00:00:00 +0000

As part of virtualization on Hyper-V, there is the ability to use Para-virtualized drivers instead of using built-in drivers for slower, emulated devices. Para-virtualized drivers are simply device drivers that are written for native virtual devices. In other words they are not emulating a pre-existing physical device. The Path to Enlightenment is simply the method you use to take an operating system and enable it to use these synthetic devices.

Installing Linux Integration Components v2.1 Beta

Wed, 31 Mar 2010 00:00:00 +0000

On March 31, 2010 Microsoft released the beta of version 2.1 of the Linux Integration Components. The Linux ICs allow Linux VMs running on Hyper-V to be “enlightened” and have access to synthetic device drivers which perform much faster than their emulated counter parts. It can be downloaded by registering for the Linux Integration Services for Hyper-V on http://connect.microsoft.com

What is new in Linux Integration Components v2.1 Beta?

There are three new features, time sync, shutdown, and SMP. Time sync is communication through the vmbus which updates the guest’s time to match the hosts (without this time is only synced on boot up of the VM). Shutdown simply makes that little shutdown button in Hyper-V Manager work, previously you only had the option of using the Stop command or entering the guest via ssh or some other mechanism and initiating the shutdown. SMP is some fixes in the vmbus module which make the module SMP-safe, and in addition Microsoft will now support Linux guests (RHEL 5.3 & 5.4, and SUSE 10 & 11) with up to 4 vCPUs, previously only 1 vCPU was supported. InputVSC (Citrix’s Project Satori) will continue to work with this release.

Enabling Jumbo Frames on Hyper-V 2008 R2 Virtual Switches

Mon, 01 Mar 2010 00:00:00 +0000

When using a Core version of Hyper-V 2008 R2, be it Hyper-V Server 2008 R2 or Windows Server 2008 R2 Standard Core, you are doing a good thing to conserve resources and minimize the attack footprint of your virtualization hosts. However this comes at a price. One of the most obvious misses in this matter is the enabling of Jumbo Frames on the Virtual Switch Interface. This enables your guests to reach out to storage directly using jumbo frames (mtu size of greater than 1500 bytes – however in this article we will be working specifically with 9000 bytes).

Hyper-V Guests: Compile Linux Kernel 2.6.32 on Debian

Sat, 19 Dec 2009 00:00:00 +0000

In this post I will be detailing the steps needed to compile a new kernel in Debian Linux 5. This particular case we are purpose building a kernel with the drivers necessary for Hyper-V Guests to take advantage of synthetic devices. Additionally I plan on expanding this series with instructions for manual compiles of other distributions as well as some pre-compiled image based installs.

DISCLAIMER: These instructions are not provided with any guarantee of support. Microsoft (company behind Hyper-V) does not acknowledge Debian as a supported guest operating system.

Hyper-V Guests: Compile Linux Kernel 2.6.32 on Ubuntu

Tue, 01 Dec 2009 00:00:00 +0000

In this post I will be detailing the steps needed to compile a new kernel in Ubuntu Linux 9.10. This particular case we are purpose building a kernel with the drivers necessary for Hyper-V Guests to take advantage of synthetic devices. Additionally I plan on expanding this series with instructions for manual compiles of other distributions as well as some pre-compiled image based installs.

DISCLAIMER: These instructions are not provided with any guarantee of support. Microsoft (company behind Hyper-V) does not acknowledge Ubuntu as a supported guest operating system. Canonical (company behind Ubuntu) will not support custom compiled kernels.

Hyper-V Guests: Linux Integration Components (v2) on Oracle Enterprise Linux

Wed, 23 Sep 2009 00:00:00 +0000

In this article I am documenting the process for installing the Hyper-V Integration Components (v2) within a Oracle Enterprise Linux 5.3 VM. My environment consists of x64 VMs, I have not taken the time to test this process for x86 VMs, however it should hold true, as long as you update the package names to reflect the appropriate architecture.

In my testing I have found that the current versions (v1 and v2) do not work with linux kernels 2.6.20 or newer (with some exceptions for v2), which means in order to use the ICs you will need to have 2.6.19 or older. Oracle Enterprise Linux 5.3 included the Linux 2.6.18-128 kernel.

Hyper-V Guests: Linux Integration Components on RHEL and CentOS

Wed, 01 Jul 2009 00:00:00 +0000

In this article I am documenting the process for installing the Hyper-V Integration Components within a Red Hat Enterprise Linux 5.3 or a CentOS 5.3 VM. My environment consists of x64 VMs, I have not taken the time to test this process for x86 VMs, however it should hold true, as long as you update the package names to reflect the appropriate architecture.

In my testing I have found that the current versions (v1 and v2) do not work with linux kernels 2.6.20 or newer, which means in order to use the ICs you will need to have 2.6.19 or older. Since both RHEL 5.3 and CentOS 5.3 included the Linux 2.6.18-128 kernel.

Hyper-V Guests: Linux Integration Components Ubuntu and Debian

Tue, 12 May 2009 00:00:00 +0000

In this article I am documenting the process for installing the Hyper-V Integration Components within a Ubuntu or Debian VM. I am specifically writing this for Ubuntu 8.04.2 and Debian 5, however I have additionally confirmed that this process works on both Ubuntu 8.10 and Ubuntu 9.04. My environment consists of x64 VMs, I have not taken the time to test this process for x86 VMs, however it should hold true, as long as you update the package names to reflect the appropriate architecture.

Hyper-V Guests: Mouse Integration on Linux VMs

Fri, 01 May 2009 00:00:00 +0000

This is a follow-up article which will allow you to get mouse control within the Guest OS without having to ‘capture’ your mouse. Now generally this is not necessary, since Linux in the Datacenter is generally without a GUI, however I use Hyper-V on my Laptop and have a Linux Desktop Guest which this makes it helpful for.

Now in order to do this you will need the linux integration components previously installed. I discussed this in a previous article for Ubuntu and Debian (found here). Additionally I plan on having a follow up article for CentOS, RHEL, and Fedora (however I am slightly slow on these since I am not as familar with them).

Saved

Mon, 01 Jan 0001 00:00:00 +0000

Search

Mon, 01 Jan 0001 00:00:00 +0000