Author: Josh Bryant

Home / Author: Josh Bryant

Creating 188 volume mount points in under 2 minutes with PowerShell and diskpart.

September 6, 2011 | Uncategorized | No Comments

Last week I showed you how I used PowerShell and diskpart to partition, format, and label 193 disks. If you’ll recall, I said I still needed to figure out how to get 188 of those disks mapped to folders for Volume Mount Points.

My goal was to get this 100% automated. Due to time constraints, I’m only at about 95%. This last bit takes a small amount of manual work. Don’t worry, it’s just simple copy and paste.

First we need to list out all volumes, to do this, open PowerShell, and run diskpart.

diskpart
list volume
exit

Next we just open Excel, put “Volume” in cell A1, and “Label” in cell B1. Copy “Volume ###” from the diskpart output and paste it into column A of the Excel sheet. Copy “Label” from the diskpart output and paste it into column B. Remove any disks that already have drive letters assigned. If you remember from last week, I had 5 disks that I assigned drive letters. After removing them from this list, I had 188 rows (not including the column names in row 1). Save it as a .csv file

Next we run a PowerShell script I called cvmp.ps1:

# Volume Mount Point Creation Script
# By Josh M. Bryant
# www.fixtheexchange.com
# Last modified 9/6/2011 8:30 AM
#
$dmount = "E:EXCH10"
$lmount = "L:EXCH10"
$data = Import-CSV C:Scriptsvolumes.csv
ForEach ($line in $data)
{
$volume = $line.Volume
$label = $line.Label
Add-Content -path C:Scriptsvolumes.txt -value "Select $volume"
If ($label -like "*L*")
{
Add-Content -path C:Scriptsvolumes.txt -value "assign mount=$lmount$label"
}
Else
{
Add-Content -path C:Scriptsvolumes.txt -value "assign mount=$dmount$label"
}
}
diskpart /s C:Scriptsvolumes.txt

That’s it! The PowerShell uses the .csv file to create the diskpart script that assigns the volume mount points.

I would eventually like to go back and script out the .csv file creation using the Export-CSV command. I was having trouble getting the output from the diskpart command (which is not a PowerShell command) into a usable format with the Export-CSV command.

Even with the manual copy and paste work, this saved a ton of time. Can you imagine how long all of this would have taken without scripting?

If you use scripts from this site, please keep the author info intact, and include a link back to this site if you post them anywhere else.

Read More
By Josh Bryant

Configuring 193 disks in under 2 minutes with PowerShell and diskpart.

September 2, 2011 | Uncategorized | No Comments

Today at work I had to partition, format, and label, 193 disks, in Server 2008 R2. Each disk was a LUN on an EMC SAN. The server is an HP ProLiant BL460c G7 Server Blade. This will be an Exchange 2010 mailbox server that is part of a Database Availability Group (DAG) when it is finished, so 94 of these disks are for databases and 94 are for transaction logs.

5 of the disks received drive letters, the remaining 188 are to be volume mount points. There was no way I was going to do any of this manually, so I created some PowerShell scripts that use the diskpart command to do all the work for me. The result is having 5 disks partitioned, formatted, labeled and drive letters assigned. Then 188 folders get created for use as the volume mount points. The remaining 188 disks get partitioned, formatted, and labeled. I’m still trying to work out a way to script mapping the volume mount points to the folders, so this is a work in progress. (UPDATED: Click here to see how I solved this problem.)

First we have the primarydrives.txt script for diskpart to create the main drives that actually have letters assigned to them:

select disk 3
create partition primary NOERR
format FS=NTFS LABEL="SAN Exchange" UNIT=64K QUICK NOWAIT NOERR
assign letter=D NOERR
select disk 4
create partition primary NOERR
format FS=NTFS LABEL="SAN Temp" UNIT=64K QUICK NOWAIT NOERR
assign letter=T NOERR
select disk 196
create partition primary NOERR
format FS=NTFS LABEL="SAN Tracking Logs" UNIT=64K QUICK NOWAIT NOERR
assign letter=M NOERR
select disk 5
create partition primary NOERR
format FS=NTFS LABEL="SAN Databases" UNIT=64K QUICK NOWAIT NOERR
assign letter=E NOERR
select disk 6
create partition primary NOERR
format FS=NTFS LABEL="SAN Exchange" UNIT=64K QUICK NOWAIT NOERR
assign letter=L NOERR

Next we have the fvmpcreate.ps1 script that creates folders based on names in a text file. I had a spreadsheet with what the database names are going to be, so I just copied those to the text file to use for this script. This script also writes a text file for use by diskpart to partition, format, and label each disk. Like I said, I’m still working on how to get it to map the volume mount points to the folders created by the script.

# Folder and Volume Mount Point Creation Script
# By Josh M. Bryant
# www.fixtheexchange.com
# Last Updated 9/2/2011 3:40 PM
#
$dbfile = "C:Scriptsdbdrives.txt"
$logfile = "C:Scriptslogdrives.txt"
$dbdata = Get-Content C:Scriptsdbnames.txt
$ldata = Get-Content C:Scriptslognames.txt
$dbpath = "E:EXCH10"
$logpath = "L:EXCH10"
$drive = 6
#
#Create Database Folders and Volume Mount Points
#
ForEach ($line in $dbdata)
{$drive = $drive + 1
New-Item $dbpath$line -type directory
add-content -path $dbfile -value "select disk $drive"
add-content -path $dbfile -value "create partition primary NOERR"
add-content -path $dbfile -value "format FS=NTFS LABEL=`"$line`" UNIT=64K QUICK NOWAIT NOERR"
}
#
#Create Log Folders and Volume Mount Points
#
ForEach ($line in $ldata)
{$drive = $drive + 1
New-Item $logpath$line -type directory
add-content -path $logfile -value "select disk $drive"
add-content -path $logfile -value "create partition primary NOERR"
add-content -path $logfile -value "format FS=NTFS LABEL=`"$line`" UNIT=64K QUICK NOWAIT NOERR"
}

The last script I called createdrives.ps1, this is the master script that calls all the others.

# Exchange Drive Creation Script
# By Josh M. Bryant
# www.fixtheexchange.com
# Last Updated 9/2/2011 3:55 PM
#
# Create primary drives.
diskpart /s C:Scriptsprimarydrives.txt > primarydrives.log
# Wait for disks to format.
sleep 30
# Create EXCH10 Folders
New-Item E:EXCH10 -type directory
New-Item L:EXCH10 -type directory
# Create Folders and Diskpart Scripts
& "C:Scriptsfvmpcreate.ps1"
# Create Volume Mount Points for Databases
diskpart /s C:Scriptsdbdrives.txt > dbdrives.log
# Create Volume Mount Points for Logs
diskpart /s C:Scriptslogdrives.txt > logdrives.log

This ended up being a huge time saver. Everything completed in about 2 minutes. Even if I can’t figure out how to script out the volume mount point mapping, it will have saved me a tremendous amount of time. The best part is this is scalable, so I can easily adapt it for use on other servers, regardless of the number of disks that need to be configured.

UPDATE: Click here to see how I solved the volume mount point creation.

If you use these scripts, or want to re-post them anywhere else, please keep the author information at the top of the script, and include a link back to this site. Thanks!

Read More
By Josh Bryant

Budget Laboratory: Part 1 – Multi-WAN Load Balancing with pfSense on VMWare Workstation

August 30, 2011 | Uncategorized | No Comments

Wait a minute, I thought this was an “Exchange blog”, why is your first article about networking with pfSense?!

Well, in order to create high quality Exchange related content for this site, I’ve been building a lab. There has been some interest in how I’ve setup my lab, so I thought I’d show you. What better way to help people learn Exchange than by showing them how to create a powerful lab environment to run it in? Besides, having a good understanding of boundary systems is very useful when troubleshooting mail flow issues.

Part 1 of my “Budget Laboratory” series is actually optional, and the only part of the lab that will cost anything (assuming you already own the hardware). It will only cost anything if you plan on using your Lab for more than 30 days.

We’ll be building most of this lab using only a single computer. I recommend a quad core CPU with at least 8 GB of RAM, however you may be able to do it with less.

The information in this guide will be useful for much more than a home lab, you could also use this in a production environment for a small business, or even a large enterprise, if scaled and configured properly.

    You will need:

  • A computer with 3 RJ45 (network) ports and a 64bit Operating System (OS).*
  • At least two high speed internet connections.**
  • VMWare Workstation 7. (30 day FREE trial, $100 after that.)
  • pfSense 1.2.3 virtual appliance. (FREE!)***
  • A Wireless Access Point, switch, or combined device.****

*If you don’t have 3 RJ45 ports available, I highly recommend adding Intel Pro 1000 NICs. You can use your on-board NIC for one of the connections. For my setup, I purchased 2 Intel Pro 1000 NICs for $30 each, and used the on-board RealTEK NIC off my motherboard for the 3rd. Why a 64bit OS? Because later on in this series I will show you how to run VMWare ESXi 4.1 inside of VMWare Workstation, and connect it to a virtual SAN. The host OS has to be 64bit for this to work. I’ll be using 64-bit Windows 7 for my examples. You can achieve the same results with your favorite 64bit Linux distribution as a host operating system.

**If you don’t have 2 high speed internet connections, you obviously won’t be able to setup Multi-WAN load balancing. However you can still setup a pfSense firewall inside of VMWare Workstation 7, and route all of your network traffic.

***I’ll be doing another guide for pfSense 2.0 when the final stable build is released for it.

****I’m using a LinkSys WRT330N, which is a combination Wireless Access Point, Router, and 4 port gigabit switch.

Step 1: Connect the modems and switch to your computer.



Multi-WAN Diagram
It should resemble the diagram above. For my setup, I have an old Motorola SB5101 cable modem that I’ve owned for years, and an SMCD3G leased modem from my cable company. This is purely for speed. If you were going to setup Multi-WAN load balancing in a production environment, you’d want to have two unique ISPs. Cable and DSL for example. That way you can have true fault tolerance and high availability. If the connection to one ISP is lost, traffic will still be routed through the other. There’s only one ISP that services the area I live in, so that’s why I have 2 connections to the same ISP. You could add additional connections to ISP(s) for even more speed or fault tolerance, just add more network ports to the computer hosting pfSense.

Each connection to your ISP should go to 1 port on the computer you’ll be setting pfSense up on. One connection goes to the switch for your LAN.

Note: If you are using something similar to the Linksys WRT330N that I have, you want to use one of the LAN switch ports on it for the connection to the pfSense firewall. DO NOT use the “INTERNET” (WAN/uplink) port, it won’t work. You also need to turn off the firewall, NAT, and DHCP server on this device, if it has those features. We’ll be using pfSense to do all of that now. You will need to change the default IP on this device as well. Usually it’s something like 192.168.1.1. We’ll be using x.x.x.1 as the gateway IP for our LAN. These devices usually aren’t smart enough to do VLANs, so if the address space doesn’t match what we configure in pfSense, things aren’t going to work. I changed mine to x.x.x.5. You can use any private address space you like for your LAN. Since mine is a small home lab, I’m using a single 24 bit subnet, so this guide will reflect that.

Step 2: Configure the host.

As I mentioned, I’m using 64bit Windows 7 as my host, so this example is from that. If you’re using a different host OS, you will need to do accomplish the same end result.

First, I recommend labeling your NICs. I gave mine names that matched what they are connected to. If I had 2 different ISPs, I would have given the WAN links the name of the ISP they are connected to, but since I don’t, I named them after the modem model number they are connected to instead.



Next we need to disable TCP/IP on the WAN links. Bring up the properties for each NIC that is connected to your ISP(s).



Make sure both “Internet Protocol Version 6 (TCP/IPv6)” and “Internet Protocol Version 4 (TCP/IPv4)”

The last thing we need to do in this step, is change the binding order of the NICs. In Windows 7 you hit “Alt+N” to bring up the hidden menu in your Network Connections, then select “Advanced Settings”. Move your LAN NIC to the top of the list.



TIP: You can disable TCP/IP on the WAN NICs from the “Advanced Settings”, just select the NIC and uncheck all check boxes for it.

This is done to make sure traffic from the host OS gets routed through the pfSense firewall. If you do not do this, both the host OS, and the pfSense VM will try to pick up an IP address from your ISP(s). Your host will end up connecting to the internet through one of these NICs and not be able to take advantage of the load balancing.

You need to leave TCP/IP enabled on your LAN NIC.

Step 3: Configure VMWare Workstation 7

If you haven’t already installed VMWare Workstation 7, do it now before continuing.

Here we need to configure the NICs for use by our virtual machines. Click “Edit” then “Virtual Network Editor”.



As you can see, I have VMnet0 bridged to one of my Intel Pro 1000 NICs, VMnet2 bridged to the other, and VMnet3 bridged to my motherboards on-board Realtek NIC. VMWare Workstation 7 doesn’t use the labels that we assigned in the host, but you can easily tell which is which. If you look in at the screen shot where I labeled my NICs in Step 2, you can see that VMnet 0 is my connection to the SB5101 modem, VMnet2 is the connection to the SMC modem, and VMnet3 is the connection to the LAN.

Step 4: Configure the pfSense virtual machine settings

Open up the pfSense virtual appliance in VMWare Worksation 7, but do NOT power it on. Bring up the Virtual Machine Settings by clicking “Edit virtual machine settings” or hitting “CTRL+D”.



Make sure you have 3 devices of the type “Network Adapter”, 1 for each NIC we configured in Step 3. Select each “Network Adapter”, make sure “Connected” and “Connect at power on” are both checked. Set the “Network connection” to “Custom: Specific virtual network”, and select one of the VMnet# that you configured in Step 3 for each “Network Adapter”.

Step 5: Configure pfSense from the console.

Now it’s time to boot your pfSense VM. When it boots, you should see a screen like this:



Your interfaces may not be assigned, mine weren’t, and pfSense was unable to automatically detect them. If this happens, it may take a little trial and error to get them assigned correctly. Press 1 then hit enter to assign them. Your WAN links should pick up an IP address from either your ISP, or your cable modem, if they use DHCP. If you get it wrong the first time, you can usually tell which one is the correct LAN interface by looking for the interface that does not pick up an IP address at the top of the screen. If you see “0.0.0.0”, that is most likely your LAN interface, write down the interface number, and run the assignment again. If you own a static IP address, you will need to configure it later. The LAN interface assignment is what is important here, once you have the correct interface assigned for the LAN, the rest of the configuration will be done through the web management console.

You can change the LAN IP by hitting 2, or use the default, it’s up to you.

Step 6: Configure pfSense from the web interface.

Open a web browser and go to http://192.168.1.1 (or whatever you changed the LAN IP to.). The default username is “admin” and the default password is “pfsense” (don’t forget to change these!). You’ll be greeted with a configuration wizard, click “Next”.



Enter your information, I used my ISPs DNS. Click “Next”
The time zone settings will come up, select your time zone and click “Next”.
Now you will asked to configure the WAN interface. If you have a static IP from your ISP, here is where you’ll configure it, UNLESS that is handled by your modem. If your modem gets the static IP assignment from your ISP, you will want to leave this as DHCP so it will pick up a private address from your modem.
The next page lets you change the LAN IP, if you did that in Step 5, or just want to use the default, you can just click “Next” and move on.
Finally we’ll set the WebGUI password. After you change it, you’ll have to log back on.

Next we need to configure the second WAN interface. Click “Interfaces” then “OPT1”.



Make sure “Enable Optional 1 interface” is checked. You can change the name here, I labeled mine “WAN2”. If you have a static IP for your other ISP connection, you’ll want to configure it here. If you only own 1 static IP, you should probably leave this set as DHCP. Again, if your modem handles your public IP, you should probably leave this as DHCP.

Next we will create a Load Balancer Pool. Click on “Services” then “Load Balancer”. When the page loads, click the icon to create a new pool.



Give the pool a name, I called mine “LoadBalancer”. Make sure the type is set to “Gateway”, and Behavior is set to “Load Balancing”.
Add both WAN IP’s to the pool. I use the gateway for each WAN interface as the monitor IP.
Click “Save” when you are done.
Create a second pool for Failover, just set the type to “failover” for this one.



You should have 2 pools as shown above when you’re done.

We’re almost done, the last thing we need to do, is setup some firewall rules. Click “Firewall” then “Rules”. When the page loads, select the “LAN” tab.



We don’t want all traffic to be load balanced. You can’t load balance SSL and other encrypted traffic, those types of connections need to use the same IP every time. pfSense processes rules from the top down. You’ll notice the load balancer rule is at the bottom. Anything we don’t want load balanced needs to be in a rule ABOVE it. I’ll show you how to create a rule to NOT load balance a protocol, using HTTPS as an example. First click the icon to create a new rule.



Your settings should match the screen shot above. Using the failover gateway we configured will ensure these protocols still work with minimal interruption if one of our WAN links goes down.

Now lets tell it to load balance everything else, you probably already had a “Default LAN -> Any” rule, just click the icon to edit it.



Your rule should look like the screen shot above. Save it, then apply the changes.

That’s it! All of your network traffic should now be flowing through the pfSense virtual machine. To verify that the load balancing is working, go to a site like www.whatismyip.com, and refresh the page a couple times. The IP shown should change back and forth between the external IPs of your WAN links every time the page loads.

If you want to test the speed out, you can use a site like www.speedtest.net, however these results can be a little inaccurate due to the “Speed Boost” that a lot of ISPs use these days. I have a 12Mbps down and 2Mbps up package from my ISP, and these are the results I get after setting up the load balancing:



If you want to test the fail over, simply unplug one of your WAN links and try surfing the web. You should still be able to even with one link down. Plug it back in, and it should automatically start load balancing again.

If I pull the plug on one of my WAN links and run a speed test, this is the result:



As you can see there’s quite a difference in bandwidth with the load balancing.

Just for fun I took a look at what World of Warcraft thought my “effective bandwidth” was…


176Mbps, don’t I wish!

I hope you’ve enjoyed this guide, stay tuned for Part 2 of my “Budget Laboratory” series, where I’ll show you how to setup a virtual SAN using freeNAS and VMWare Workstation 7.

Read More
By Josh Bryant