Solaris 11 on the bin

SPARC Logical Domains: Alternate Service Domains Part 3

Wed, 17 Dec 2014 00:00:00 +0000

In Part One of this series we went through the initial configuration of our Logical Domain hypervisor, and took some time to explain the process of mapping out the PCI Root Complexes, so that we would be able to effectively split them between the primary and an alternate domain.

In Part Two of this series we took the information from Part One and split out our PCI Root Complexes and we configured and installed an alternate domain. We were also able to reboot the primary domain without impacting the operation of the alternate domain.

SPARC Logical Domains: Alternate Service Domains Part 2

Tue, 16 Dec 2014 00:00:00 +0000

In Part Two (this article) we are going to take that information and split out our PCI Root Complexes and configure and install an alternate domain. At the end of this article you will be able to reboot the primary domain without impacting the operation of the alternate domain.

SPARC Logical Domains: Alternate Service Domains Part 1

Mon, 15 Dec 2014 00:00:00 +0000

In this series we will be going over configuring alternate I/O and Service domains, with the goal of increasing the serviceability the SPARC T-Series servers without impacting other domains on the hypervisor. Essentially enabling rolling maintenance without having to rely on live migration or downtime. It is important to note, that this is not a cure-all, for example base firmware updates would still be interruptive, however minor firmware such as disk and I/O cards only should be able to be rolled.

SPARC Logical Domains: Live Migration

Tue, 09 Dec 2014 00:00:00 +0000

One of the ways that we are able to accomplish regularly scheduled maintenance is by utilizing Live Migration, with this we can migrate workloads from one physical machine to another without having service interruption. The way that it is done with Logical Domains is much more flexible than with most other hypervisor solutions, it doesn’t require any complicated cluster setup, no management layer, so you could literally utilize any compatible hardware at the drop of the hat.

Solaris 11: Configure IP Over Infiniband Devices

Mon, 08 Dec 2014 00:00:00 +0000

In this article we will be going over the configuration of an infiniband interface with the IPoIB protocol on Solaris 11, specifically Solaris 11.2 (previous versions of Solaris 11 should work the same, however there have been changes in the ipadm and dladm commands).

Identify Infiniband Datalinks

First we need to identify the underlying interfaces of the infiniband interfaces. In my case net2 and net3.

 bash
 
 
dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown ixgbe1
net0 Ethernet up 1000 full ixgbe0
net2 Infiniband up 32000 unknown ibp0
net3 Infiniband up 32000 unknown ibp1
net5 Ethernet up 1000 full vsw0

Another way to confirm the infiniband interfaces is to use the show-ib command.

Adventures in ZFS: Mirrored Rpool

Mon, 01 Dec 2014 00:00:00 +0000

It always makes sense to have a mirrored rpool for your production systems, however that is not always how they are configured. This really simple procedure is also critical.

Create a Mirrored Zpool

Check the existing devices to identify the one currently in use.

 bash
 
 
zpool status rpool
 pool: rpool
 state: ONLINE
 scan: none requested
config:

 NAME STATE READ WRITE CKSUM
 rpool ONLINE 0 0 0
 c0t5000CCA0436359CCd0 ONLINE 0 0 0

errors: No known data errors

Once we know which one is currently in use, we need to find a different one to mirror onto.

Solaris Networking: Datalink Multipathing Protocol

Mon, 19 May 2014 00:00:00 +0000

In this article we are going to go over Datalink Multipathing Protocol (DLMP) available in Solaris. DLMP is similar to IPMP, however there are some key differences, the biggest being the layer at which it operates. DLMP operates at the datalink layer of the OSI model, while IPMP operates at the network layer. Due to the differences in those layers DLMP opens up a lot of possibilities that were not possible with IPMP. For example if you had a requirement for redundant networking for a service, either IPMP and DLMP would both be able to meet that requirement very well. However if you had a requirement that the service runs inside of a zone or a logical domain then the level of work in IPMP becomes much higher. This is because these hypervisors assign datalink devices to their guests. Since the redundancy for IPMP is built on a level higher we need to assign multiple non-redundant interfaces to the guests and then build the interfaces and IPMP groups inside each of the guests.

Solaris SMF: Define File Dependency

Thu, 15 May 2014 00:00:00 +0000

One of the biggest benefits of migrating to the Service Management Framework is that we can introduce dependencies to a services, these dependencies can be other services which is very valuable, however they can also be file based. This can be used in a number of ways, say for example you have some application trees that live on some NFS mounts, in the event that the mount hasn’t been successfully mounted it will still attempt to start the service. However if we make our service dependent on the file, then if the file doesn’t exist it won’t even attempt to start the service.

Solaris SMF: Define Service Execution User

Wed, 14 May 2014 00:00:00 +0000

Frequently we want a service to execute as a non-root user, this is pretty trivial in the context of the Service Management Facility in Solaris. This article will go over exactly what goes into that. The one complexity around that is the environment, if you have environment variables that the user will depend on those, they will need to be setup inside of the SMF. I will not be going into that in this article.

Solaris SMF: Delegate Service Control

Tue, 13 May 2014 00:00:00 +0000

In many of the environments that I deliver to a client we have a concept of an “environment account” this account is the credential set that the environment runs as. Over time and the maturation of the Service Management Facility (SMF) we have been able to simplify this offering and increase its capabilities. Today we are going to talk about allowing the environment user to manage a service controlled by SMF.

ZFS: Send and Receive

Tue, 24 Sep 2013 00:00:00 +0000

ZFS gives us the ability to move data with ZFS send and receive. This can be used to combine snapshots, create full or incremental backups, or replicate data between servers on your LAN or even over the WAN.

Local Backup with Send

This is very simple, simply perform a send of a snapshot then output it to the file, I have complicated it a bit to include a folder which is named off of the current date (as would be helpful with backups).

Solaris Networking: IP Multipathing Protocol

Tue, 10 Sep 2013 00:00:00 +0000

In this article we are going to go over some basics of using IPMP in Solaris. We will discuss this on the latest versions of Solaris 10 and 11 on the SPARC platform, these instructions should also work on the x86 platform, but will be dependent on the listed version numbers, as the networking stack has changed alot of the past few minor versions of Solaris 10 and 11.

Solaris 11.1

Create an IPMP Group

Adventures in ZFS: Deletion of a Recursive Snapshot Set

Mon, 03 Jun 2013 00:00:00 +0000

Quite often I will find it necessary to create a recursive snapshot over a large number of ZFS file systems. In this case, I find that frequently have little long term need for them, and need a quick and easy way of disposing of the snapshots created without spending large amounts of time manually deleting all of the snapshots.

Creating Some Snapshots

Here we will create some test snapshots. Make sure you do this on a test system, in a throw away area, this way if something is mistyped it doesn’t cause any problems.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part Two

Tue, 07 Aug 2012 00:00:00 +0000

In our previous article we went through the process of preparing a Solaris 11 for SPARC machine as a Logical Domain hypervisor. Now that we have laid out our foundation we are ready to create some Logical Domains.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Create the Logical Domain

bash

ldm add-domain t4-g1

Configure Devices

We will need to add devices to our newly created domain. Here we will add virtual CPUs, memory, and a network device.

Solaris Virtualization: Using Logical Domains on Solaris 11 Part One

Mon, 06 Aug 2012 00:00:00 +0000

Solaris 11 for SPARC, as well as previous versions of Solaris, include a neat little technology called Logical Domains (which has been rebranded as Oracle VM for SPARC). Logical Domains are implemented in the SPARC processor.

This series is going to focus on the basic administration of a Logical Domains environment using the basic command line tools of the Solaris 11 Operating System. In Part One we will go over what it takes to configure a Solaris 11 installation to be a Logical Domains hypervisor. In Part Two we will document defining and configuring Logical Domains. In Part Three we will discuss connecting to the Logical Domain in order to perform an installation of the operating system.

Solaris 11: DNS Client Configuration Using Svccfg

Wed, 23 May 2012 00:00:00 +0000

In Solaris 11, alot of configurations are being moved from configuration files into the Service Management Framework. Here we will discuss this change around the DNS client.

View Existing DNS Client Configuration

 bash
 
 
svccfg -s network/dns/client listprop config
config application
config/value_authorization astring solaris.smf.value.name-service.dns.client
config/domain astring test.local
config/nameserver net_address 10.0.0.152

Update Existing DNS Client Configuration

Here we will update our name servers. In this case we are replacing the original with two different addresses.

Solaris 11: Change Hostname Using Svccfg

Tue, 22 May 2012 00:00:00 +0000

When Solaris 11 was released more configurations were moved into the service management framework, and moved out of configuration files, one of these changes was the hostname/nodename.

View Hostname in SMF

This will read back the configuration out of the SMF.

 bash
 
 
svccfg -s system/identity:node listprop config
config application
config/nodename astring solaris11box
config/loopback astring solaris11box

Change Hostname in SMF

We have two separate settings, which need to be updated.

Adventures in ZFS: Configuring iSCSI Targets

Mon, 27 Feb 2012 00:00:00 +0000

The most critical thing about any storage implementation is flexibility. This means that you need to be able to adapt your solution to changing environmental considerations. The nice thing about a ZFS-based solution is that all of the building blocks are available. If you want to use it as a NAS device, you can use CIFS, NFS. If you want to provision Fibre Channel that is available too. Even some of the more sought after SAN features are included, replication, deduplication, thin-provisioning, and many more. Now in todays article we are going to discuss how you can go about setting up iSCSI targets. I personally am not a fan of iSCSI (I have been bitten by the hard knee too many times), however you really can’t argue with the flexibility that is provided by being able to quickly provision storage on already existing or readily available hardware. The best thing about this process is that it is really similar to provisioning fibre channel targets, so if you are familiar with this process (available here) then you will be able to quickly adapt to the iSCSI variation.

Adventures in ZFS: Time Slider is not Expiring Snapshots on Imported Zpools

Tue, 06 Sep 2011 00:00:00 +0000

A few days ago someone contacted me with an interesting issue with regards to the ZFS automatic snapshots, facilitatedby the time slider service.

The Situation

OpenSolaris machine with zpool exported. Time slider was working prior to export.
Solaris 11 Express fresh install with zpool imported.

The Symptoms

Basically the core of the issue was that the zpool was from an OpenSolaris machine, in lieu of doing an upgrade from OpenSolaris to Solaris 11 Express, the zpool was exported and imported into a machine which was already running Solaris 11 Express. Upon the import new snapshots were being created by the time slider service however old ones were not being expired. You can still manually delete snapshots.

Adventures in ZFS: Configuring Fibre Channel Targets

Wed, 24 Aug 2011 00:00:00 +0000

In our environment ZFS has become a critical component of our Storage Infrastructure. We have been able to provision Fibre Channel storage from our ZFS file systems which give us the benefit of data integrity, deduplication, performance through use of the ARC, L2ARC, and ZIL as needed. Additionally the real benefit to ZFS is our storage can run on commodity hardware which reduces the TCO significantly over the major Storage vendors. Now administering ZFS is really quite simple, however getting Fibre Channel can be a bit more work. However after reading this you ought to be able to handle this with your eyes closed, well maybe not with your eyes closed. But you will be much better prepared.

Adventures in ZFS: Faulted Cache Devices

Mon, 15 Aug 2011 00:00:00 +0000

We recently had an issue with Crucial M4 Solid State Disks when using them with ZFS on Solaris 11 Express (snv_151a). Basically the disks were showing a whole bunch of write errors and had been “FAULTED” by ZFS. Now to make this problem even worse when we tried to zfs clear them it locked up my SSH session, as well as subsequent sessions, it would allow me to initiate subsequent sessions (and authenticate), however would not deliver me to the prompt. Additionally when I went onto the console if I ran a “zfs list, zpool list, or zpool status” it would lock up the shell every time. When I say lock up, I mean lock up, no CTRL + C or Z or anything. So since these devices were cache devices only this wasn’t the end of the world for us.

Solaris 11: Mount SMB Share Locally

Tue, 31 May 2011 00:00:00 +0000

Currently I have been working on a method of migrating some of our data from Windows file servers to Solaris ZFS CIFS servers, of course trying to retain as much feature parity as possible. Permissions are an issue, however I have worked through most of those issues (look for this article in the future), now it is time to migrate data from Windows onto ZFS. This first crack will be about 1TB of data. Enter rsync. The “easiest” way to do this would be to use a Linux box in the middle to mount both the Solaris CIFS share and with Windows CIFS share and perform the rsync from there. A far more efficient way would be to mount the Windows CIFS share on the Solaris box or vice versa and perform the rsync between the two boxes. This article will document how to mount a Windows CIFS file share onto the local file system of the Solaris 11 Express box.

So You Want to Learn ZFS Part One

Mon, 04 Apr 2011 00:00:00 +0000

Enterprise Storage is currently broken. IT departments big and small spend way too much money on completely ineffective storage solutions which only partially fulfil the companies storage need. All the while having to justify massive expenditures to the business which do not meet the need. Essentially IT departments everywhere have become storage sales reps, sure we work with vendors who provide us with quotes, but when the rubber meets the road it is the IT department which is selling the storage to the business. If any of you have ever been in this position as I have and literally felt sick to your stomach because you knew it was all wrong, then this article is for you. This article does not provide an Enterprise Storage cure-all. What it does provide is an interesting look at a solution which really revolutionises the entire Enterprise Storage model. More importantly it allows you the flexibility, features, and densities needed to make it far superior to any modern day Enterprise Storage solution. Also it is pretty darn cheap.

Solaris 11: Join AD Domain for Samba

Wed, 30 Mar 2011 00:00:00 +0000

Configure Kerberos

Make a backup of our original, just in case.

bash

cp /etc/krb5/krb5.conf /etc/krb5/krb5.original.conf

These sections already exist in the file so you will want to replace them with the correct values for your environment.

 bash
 
 
vi /etc/krb5/krb5.conf

[libdefaults]
default_realm = ALLANGLESIT.COM

[realms]
allanglesit.com = {
kdc = dc.allanglesit.com
admin_server = dc.allanglesit.com
kpasswd_server = dc.allanglesit.com
kpasswd_protocol = SET_CHANGE
}

[domain_realm]
.allanglesit.com = ALLANGLESIT.COM

Quick recycle of the services or a start if they aren’t running.

Solaris 11: Network Configuration Advanced

Tue, 29 Mar 2011 00:00:00 +0000

In my previous post we went over the basics of configuring a network interface in Solaris 11, without using the Network Auto Magic (NWAM) which is enabled by default. In this article we will go over some of the more advanced features which can be leveraged including VLANs, aggregation groups, and jumbo frames.

Configure VLAN Tagging

Create a VLAN, specify the VLAN id with the “-v”, then specify the interface with the “-l”, and finally create a name for the tagged interface (in this case user0 since this particular system is the user VLAN, if you have more descriptive names of your VLANs you can use them here).

Solaris 11: Network Configuration Basics

Mon, 28 Mar 2011 00:00:00 +0000

I have been doing research for my next big series “So You Want to Learn ZFS.” This series is basically going to be a multi-part series of How-To’s which hopefully will give you the ability to build a file server (or even a SAN) based on ZFS if you so choose. However there are a few things that I failed to take into account.

Solaris 11 is so different from Solaris 10.
Solaris 11 is so different from EVERYTHING else.

I figured that I’d be able to kind of gloss over the high points of how to get your system up and running and just dive right into the fun ZFS stuff. So before we get into the good stuff there are some basics that we will need to go over first. Today we will cover basic networking.