- Building a Windows- iSCSI Dram Based SSD SAN Guide -

Purpose

As you will read below, most SSD's (Solid State Drives) either have their own weaknesses or are too expensive to warrant the purchase. We will essentially be making our own Dram based SSD that is shared through iSCSI which allows for permanent data persistence, great access times, and gives greater throughput than any competition. The applications of this will allow you to install your favorite application on the drive and get near instant loads and response times, or perhaps used to hold your page file, Photoshop scratch disk...etc. Because the drive can provide excellent throughput plus excellent access times, nearly any application can benefit. Databases mainly get an improvement from lower access times, programs like 3ds Max and Photoshop would get an improvement from both. So, no matter how you choose to use it, you can expect at least some improvement.

Whats Available?

Hard drives are a form of permanent digital storage. Every computer user uses a hard drive whether or not they realize it. A hard drive is used to store the Operating system (Ex. Windows, Linux, OSX), your programs, and all of your data.
However, hard drives have the disadvantage of being considerably slower than your computers "RAM," a faster but temporary storage space. As such, if data is being requested it will always be loaded off of the hard drive and into the systems RAM for actual use. However, the problem is, the difference between an average, high data density, mechanical drive is 50-70 MB/s transfer speed compared to 2,000 MB/s for ram! Even worse is the access time difference. Access time is the time it takes for a file request to be fulfilled, this means the entire process of requesting a file, finding it, reading it, and transferring it into ram. The difference can be 6-15 ms for a mechanical drive compared to 0.05-0.1 ms for ram!
Again, a very large difference.
Now, there has been new technologies developed to address certain aspects of this:
1.)Solid State Drive- Type 1: Solid State Drives (SSD's), these are drives that use Integrated Circuits (IC's) to record and store data. There are two types. Type 1 uses NAND, and similar chips to store data. These SSD's are very similar to "flash", "thumb", "usb"... etc drives and use the very same technology. Here the chips record data by injecting electrons into cells, a full cell is a value of 0, an empty cell is a value of 1. This type of drive acts as a permanent, non-volatile, storage. If the power is lost, the data is still there (though it will fade over a large space of time, read years upon years). However there are a few problems:

Pros:
-Read speeds are the same or better than a mechanical drive, particularly those that are reads for small random files. Similar throughput throughout all the storage space, due to no moving parts, and as such no differences in angular velocity.
-Access Times on type 1 range from around 0.1 to 1 ms

Cons:
-Writes generally tend to be slow to NAND memory chips, partially due to erase blocks being large.
-Higher price per gig, and consumer available drives are not as large compared to available mechanical drives.
-There are a limited number of times each cell can be written too before it is damaged. This is a common problem, and as such intelligent firmware will generally spread out the cell writes. For example, you write to one cell, then even if you attempt to write to that very same cell, the write will be carried out on the next cell. However, cell damage still occurs over time, giving rise to a growing table of bad cells, and having data relocated, eventually the size of your drive will shrink as more cells are marked bad.
-Higher long-term power consumption. While SSD's wattage usage may be lower, it consumes that much all the time, compared to a mechanical drive that only barely uses its max wattage in select situations. The result is that you may use more power over time compared to a mechanical drive.

2.)Solid State Drive- Type 2: Here we have a similar idea to the type 1 drive, but these use DRAM chips, that is, dynamic ram chips. Dynamic ram chips require power refreshes in order to actually retain the data contained within them. A hard drive is made using these chips, and is usually composed of the chips, the storage controller and its interface, and under most practical cases, a permanent storage backup.

Pros:
-Read and write speeds are extremely fast, throughput is usually limited by the interface.
-Suffers no write number limitation.
-Access times are quite low, around 0.01-0.1, depending on the rams' timings.

Cons:
-Data is lost on DRAM based storage solutions if the power is lost. This is remedied by combining DRAM storage with permanent storage backup, and a battery. If power is lost, the DRAM will operate on the battery long enough to move all data to the permanent storage backup, when power is restored all data is moved back to the DRAM.
-Cost per gig is amazingly huge, compared to NAND based SSD's and conventional mechanical drives. Along with this, most SSD setups that contain both DRAM and a practical backup system is even more expensive.
Generally this type of drive is not a consumer level product, almost all consumer SSD offerings are NAND based from what can be observed in most consumer outlets.

3.)GigaBytes I-Ram: This, to my knowledge, is a one of a kind product, being a consumer level DRAM SSD offering (compared to the HyperDrive which goes for a couple thousand). It is a pci peripheral card that supports up to 4 gigs of ddr ram, and is combined with a battery backup. It only uses the pci slot to provide power while it is running, and to charge the battery. The device actually interfaces through a SATA I interface. SATA I is limited to 150 MB/s theoretical, which is around 140 MB/s or so in real life application and provides nice access times, giving around 0.05 ms. The I-Ram will set you back around $400 for the total package of the card and 4 gigs of ddr1.

Pros:
-Great access times.
-Can last around 16 hours on battery backup before any data is lost.
Extremely fast read and write times, which are consistent throughout all of the storage space.

Cons:
-Interface choice bottlenecks max throughput to a theoretical 150 MB/s.
-Only backup is a battery backup. If the power goes out longer than the battery backup can sustain, all data will be lost. The battery backup also, like all others, may slowly degrade in capacity, and as such may last even less without power over time.
-The use of ddr is more expensive per gig than ddr2.

4.)Ram drives: This is a software solution, with all of the benefits of Gigabytes I-Ram, except for there is no interface limitation. In this case a portion of your system ram is reserved and used as a drive. The benefit is that you get the full potential of the ram throughput, as you're using your ram that is actually part of your system. But it also has many downsides.

Pros:
-Extremely fast, read and write speeds are those of your ram, and access times are excellent.

Cons:
-Uses your system ram, so if you have 1 gig of ram (1024 MB), and create a 512 MB ram drive, you now only have 512 MB of system ram.
-Increase is processor (CPU) utilization as the ram drive is being used. This affects overall system performance as processor clocks are diverted to deal with ram drive data exchanges.
-No battery backup for just the drive. If power is lost, all data is gone.

Our Drive

Our drive will be using Cenatek's RamDrive software, Windows Vista x64 (basic), a 275 watt power supply, AMD Sempron 3400+ single core, MSI motherboard with Nvidia chipset and onboard video, Apex micro-atx case, Western Digital Caviar 40 Gig drive, and 8 gigs (4x2 gig sticks) of Kingston ddr2 800 ram, 2x 1 gigabit network cards that use the intel pro server nic's (required for connection "Teaming"). The point is to build a slim, efficient, low-power computer for which to run the needed software. The small form factor was chosen to produce a sleek, unobtrusive case for which to place onboard on the main computer (in this case, a fullsize server case). And the total price is the same as the Gigabyte I-Ram but offers quite a bit more benifits.

The main difficulty with a project like this is the interface between one computer to another, given the large actual throughput potential given by the ram. As such, the best low cost solution i could come up with is 2, 1 gigabit, network interface cards per computer. These connections are then combined to act as a single "logical" Ethernet connection, giving a theoretical 2 gigabit total throughput (2x1000= 2,000/8= 250 MB/s).

Theoretical function:

Basically, we will create our computer, install all of the components and the OS, then move on and install Cenatek's Ram Drive software. The software will take any portion of the ram that is not in use by the OS, and create a fixed disk based on that ram. We'll install Rocket Div.'s Starwind on the host computer (the ram drive computer), which will allow us to share the ram drive by having it act as an iSCSI device. The iSCSI device will share the drive over a two-cable Ethernet interface, derived by two nics and two cross-over cables that are combined to act as a single logical Ethernet interface. This interface will carry communications back and forth by having the client computer, running Rocket Div.'s Starport, which mounts the drive as a fixed disk. The client computer itself is accepting the two separate cross-over cables via 2 nics that too are combined to form a single, logical Ethernet adapter. Now, what are the pros and cons of this setup?

Pros:
-250 MB/s theoretical transfer rate. Because the interface is the limiting factor, you can expect both read and write speeds to be 250 MB/s
-Ability to easily expand the throughput. When i say 250 MB/s i mean for this project, that is using 2 gigabit network cards. You can very easily upgrade the throughput to 500 MB/s or higher by adding more cards and "Teaming" them.
-8 Gigs possible now with possibility of expansion. You're only limited by how much ram your motherboard supports, and motherboards are interchangeable. Later down the road, you could upgrade and go higher if need be.
-Battery backup can be given to the computer. This can act as a battery backup for the ram drive contents.
-No battery backup may actually be needed based on drive usage. If the drive is only meant to contain "temp" files, such as a Photoshop scratch disk, paging file, install files, or files meant for security, then no battery back up is even needed. However, if the drive is used for an application install (such as Photoshop), Cenatek's Ram drive software contains a periodic backup option. You can set this option to every 300 seconds and up. This backup saves an image of the current ram drive, and is restored at startup. So, even if power is completly lost, for months, years...etc, the drive's contents will be restored when the computer is started back up.
-A major increase in throughput over I-Ram, from 150 MB/s to 250 MB/s theoretical.

Cons:
-Overhead from iSCSI, TCP, and other software increase the access times to 0.7 ms on average. However, it is arguable if this increase would even be noticeable.
-Due to the drive being Software iSCSI, to my knowledge, the drive can not be made to be a boot drive.
-May use more power than other SSD's, although measures can be taken. For example, all extra peripherals can be disabled, the processor can be underclocked with a reduced voltage (after all, not that much processing power is required to handle ram drive requests).

Getting Started

1.)Build the computer based on your chosen parts. If you are not sure how to build a computer, search google, there are a ton of great articles that can assist you.
2.)Install your Operating System. If you have 3 gigs or under, that you wish to use for your ram drive, i recommend just using plain Windows 32 bit. However, anything over 3 gigs i would recommend A 64 bit version of Windows. This is because, 32 bit OS's can only reference 32 bit memory address space, which just happens to be limited to 4 gigs of ram. However, a "memory hole" exist for your peripherals to be mapped, eating up ram. This usually leaves only 3.5 gigs of ram for use, with a limit of 2 gigs per application that is not large address aware. A 64 bit OS greatly increases this limitation to give reference space of up to 16 terabytes of ram, and 64 bit motherboards give an option to "remap the memory hole". However, Windows limits the max ram usage based on what version, even if it is a 64 bit OS, so make sure your Windows version supports the amount you're wanting to use.
3.)Download and install Cenatek's Ram drive software, and install it on the ram drive host. You can download a free trial version, but you'll eventually have to buy the software if you wish to create a ram drive over 400 MB in size. However, it is extremely low cost at $24 for every version except for the server edition. You can also use the free ram drive software that is part of Starwind, however you loose the drive image backup and restore feature, if this is your choice then you must only use the ram drive for temp files and nothing more as the data will be gone after the computer is turned off.
4.)Download and install Rocket Div.'s Starwind on the ram drive host computer. This is the iSCSI "target" software, this allows you to create a shared iSCSI device. The software is a bit expensive, however, if you wish to only create one iSCSI device to share, you can download and use the "personal" edition, which is FREE.
5.)Rocket Div.'s Starport on the client computer. This is the iSCSI "initiator" software that allows you to connect to your shared iSCSI device. Again, if you don't need any of the extra features you can download Starport "personal", which is free.

6.)With Cenatek's software installed on the ram drive host computer, go to Start -> All Programs -> RamDisk* (*RamDisk version) -> Cenatek Configuration Tool:

(Screenshot of RAMDiskXP)

This is the screen you'll be greeted with. Here you can choose you ram drive size and partition type. However, i recommend setting to "unpartitioned" and using Windows Disk Management to format and configure the drive. On the next tab you have these options:

Here you can choose your ram drive image backup options. If you wish to use the backup option you need to check "Load Disk Image at Startup", "save Disk Image at Shutdown", and "Auto save." I would not recommend using background update as the ram disk will be limited by the speed of the hard drive (by throughput), but this would be the safest option to choose as you're guaranteed to not have any data loss. Though if background updating is absolutely needed you may want to look into setting up a raid to do the background backup while still allowing for moderate throughput. The auto save function will periodically save an image of the ram drive, try to set this to the highest time you can, and yet insure your data is uptodate. It may take some experimentation to figure out the best time. During each auto save pass, the ram drive driver will slowdown for a very short period of time to service the save request.
To configure the drive once it has been created, right-click on "My Computer", go to "Manage", click on "Disk Management":

From here you can format the drive and give it any drive letter you want. I would recommend FAT32 for any ram drive under 2 gigs and NTFS for anything above 2 gigs. FAT32 can provide less overhead around that mark as apposed to NTFS, however NTFS is faster than FAT32 Above that mark. However on a ram drive that may be a moot point, but it still may increase the final access time nonetheless.
7.)After your ram drive is created you need to now share it through Starwind. However, Starwind, in my experience will not recognize Cenatek's ram drive if you try to do a straight "Disk Bridge Device." So a little trickery will have to be used. On the ram drive computer go to Start -> All Programs -> Rocket Division Software -> Starwind -> Click on Starwind. Right-click on "Connections" and click on "Add a Connection," create a "localhost" connection and keep the port setting the same, change any of the security settings if you wish to use CHAP (security protocol), hit "OK." Now right-click on the new connection and hit connect. From here you can click on the connection and hit "Add Device" from the toolbar at the top of the window. At this screen you can choose to use Starwind's built in ram drive by choosing "RAM drive device" and click next, choose your size and partition/file system options, enter the name of the device and then hit next and then next. However, if you choose to use the built in ram drive device you will loose the backup options that Cenatek gives you, but it may be a good solution if your use does not require the data to persist after you restart your computer or if the power goes out.
If you choose to use Cenatek's ram drive software you will instead have to click, "Add device," choose "Image File device," click next, choose "Create new image," choose the size (size of your ram drive), make its location on your main system drive (most likely c:) and finally give it a name and place the extension .img after the name, click next and click next. Give the image a device name, click next and next. Now, go to your system drive, find the image file, right-click on it and choose "Copy," go to your Cenatek ram drive, right-click and choose "Paste." Go back to Starwind, click "Add device" once again, choose "Image File device," click next, this time choose "Mount existing image" and click next, click on the button to the right of the address bar and browse to the .img file and select it and hit "Ok." Then click next, type in a device name, then click next and next.
Now you have officially created an iSCSI shared drive that is actually made from your ram. Now, its time to network the two computers together, the host and the client, and attach our newly created drive on the client.
8.)Just for testing purposes, for now just use one network card per computer. Connect one to the other through a single crossover cat5 cable, using two unused network cards. It is imperative that the network cards you use to interface the two computers are dedicated, meaning they are not used for any other transfers. Otherwise access times and throughput for the drive will increase and decrease, respectively, as the two transfers battle over the available bandwidth.
From here, you'll want to go, on the host computer, to start -> Settings* -> Control Panel -> Make sure you're in classic view -> Network Connections -> Right-Click on your network connection and go to properties -> Double-click on "Internet Protocol(TCP/IP)" -> Choose "Use the following IP Address" -> Type 192.168.254.77 for the IP, 255.255.255.0 for the subnet, and ignore the default gateway since this is a dedicated connection, same with the DNS section, ignore this too. Now, do the same for your client computer, but this time give it the IP Address 192.168.254.78 and a subnet of 255.255.255.0, again, you can ignore the rest.
9.) Now, on the client computer go to Start -> All Programs -> Rocket Division Software -> Starport -> Click on Starport. Click on the computer icon that uses your computer's name, click "Add Device", choose "Remote iSCSI device" and click next, to make it easy for yourself just use the host's IP Adress in the "Remote iSCSI device IP Address or Machine Name" window (if you followed the same numbers i gave it would be 192.168.254.77), leave the port number the same and click next, the ram drive you created should be shown under the target list, select your target, configure any of the CHAP settings if you configured CHAP on the host and click next and then finish.
Your ram drive that resides on your host now is recognized as a fixed disk on your client, but you need to configure it. Once again, right-click on "My Computer" and choose "Manage", select "Disk Management", you now should see a new drive that is the same size as your ram drive. You'll have to right-click on the disk number and say "Initialize Disk", then right-click on the Unallocated space to and choose "New Partition", then follow the instructions.
9.) From here you should be able to access the drive and actually use it. However, your throughput and access times will be limited until we install the other network cards and use intel's "Teaming" feature to provide link aggregation, giving 2x 1 gigabit throughput.
Again, you really should use only dedicated networking for the drive. Here is a screenshot showing the drive over a busy, already in use, connection. Since the interface is the bottleneck in this situation and not the device, the throughput should always be a straight line, constant. Here you see various peaks and dips, this is because other applications are competing for throughput and priority. That is why i recommend multiple nics using direct connection crossover cables that go directly from one pc to another (which also takes out the hub/switch from the equation of possible problems and troubleshooting).

For more information on what teaming is, please read the following quote from Intel's doc's:

"Advanced Network Services (ANS) Teaming, a feature of the Advanced Network Services component, lets you take advantage of multiple adapters in a system by grouping them together. ANS teaming can use features like fault tolerance and load balancing to increase throughput and reliability."
If the throughput of one network adapter is reached, any additional data, will be routed through one of the other network cards.

The mode we're interested in is Static Link Aggregation (SLA):"provides increased transmission and reception throughput in a team of two to eight adapters."

To create a team, please follow this tutorial here, just choose "Link Aggregation" instead of "Fault Tolerance".

10.)From here you should have a working, fast, external drive that performs like a fixed disk on your computer. All that is left is for you to enjoy. But if you're like me you'll attempt to optimize the drives interface to maximize throughput and minimize access times. I recommend using the program HD Tune for testing. This application can also test raw file systems, so no need to even format the drive for testing. As far as how to optimize your network interface, that subject is beyond the scope of this tutorial, and could easily require its own article to cover the basics. But google gives access to a plethora of websites that can assist you.

Good luck and have fun!
Jarrod Christman