With data burdens for organizations increasing at dizzying rates, storage management has become more important than ever. That wasn’t been lost on Microsoft in its continuing development of Exchange Server 2010. The software maker has applied a new philosophy to how the application handles storage. It takes into account the declining price of storage and the pressure to improve performance across the storage infrastructure. It embraces using direct-attached storage instead of disk arrays, continuous replication to spare servers instead of RAID or clustering and cheap disk arrays as a substitute for tape backups.

Exchange 2010, as did Exchange 2007, has improved its handling of input/output loads for a given number of simultaneous users. One way it does that is by shelving a technique for storing copies of email messages that Microsoft has used in all previous versions of Exchange. In those versions, Exchange tries to store all copies of a message at a single location on disk. That saves disk space but reduces performance. Exchange 2010 stores copies wherever there’s free space. That may eat up more space, but Microsoft felt the performance improvement was worth it. Those kinds of improvements in Exchange 2010 opens the door for IT departments to use more economical alternatives to traditional and expensive solutions, such as substituting serial attached storage for network attached storage.

While in economy mode, an IT department may be inclined to choose SATA rather than SAS drives. That may not be a wise idea in a high performance environment. SAS drives use a technology called Tagged Command Queuing (TCQ). It copies better under heavy loads than SATA drives, which use Native Command Queuing (NCQ). Both technologies improve performance by moving the management of I/O communication with a platter to the hard drive itself. The queue holding requests for hard drive action in NCQ devices, however, tops out at 31 requests. No such limitation is set on TCQ devices. In addition, SATA drives are built to a price point so they tend not to be as reliable as SAS drives. Moreover, the price differential between the two technologies isn’t significant when “peace of mind” considerations are factored in. According to Microsoft, SAS drives cost five percent more than SATA drives, yet perform 25 percent better under peak loads.

Deployment of RAID can be avoided, too, thanks to Standby Continuous Replication. It allows each server storing mail messages to be replicated on a hot-spare server. What’s more, the technology works over long distances so the hot-spare can be located at a distant location, which will make disaster recovery planners very happy. The approach can be very resilient. Microsoft, which has implemented the approach in its own operations, has found that over the three year life span of a server, an average of one drive will fail. When a drive fails, the database on that drive fails over to a replica server. According to Microsoft, its system experiences less than an hour of downtime per year.

Exchange 2010′s new storage technologies can also provide an escape hatch from total dependency on tape for backup storage. A major challenge of IT administrators is trying to cope with shrinking backup windows. As data grows, it takes longer and longer to back it up. Couple that with an expanding demand for availability due to business hours expanding from nine to five to 24/7 and some IT honchos can find themselves in a real bind. Of course, more hardware could be thrown at the shrinking backup window problem, but the kind of iron needed to meet past levels of performance can be expensive–both in terms of the kind of tape drives required and the media needed to feed the drives.

To some extent, backup problems are partially addressed by replication of data to the hot-spares. The difficulty there is there are no historic backups. If the backup on a hot-server gets corrupted, there are no earlier backups from which to restore the data. Microsoft’s answer to that was to create another level of nearline storage manned by SATA drives configured in a relatively inexpensive RAID 5 array.

The backup regimen for the array depends on an organization’s priorities. Microsoft, for instance, does an incremental backup every 15 minutes and a full backup overnight. That allows them to keep at least a week of backups on hand should something go wrong. Since even incremental backups can degrade system performance, the company makes its backups from the hot-spares. That diverts load demand from the live databases that are online and utilizes an under worked resource–the hot-spares.

Exchange 2010 has some storage twists

Free ebook download: Top 10 Most Popular Troubleshooting Posts for Email Administrators

Ten years ago we measured mailbox sizes in megabytes.  A 20mb mailbox was adequate.  A 100mb mailbox was a luxury.

Today we measure mailbox sizes in gigabytes.  A single message in today’s email communications could easily consume the entire mailbox quota of a decade ago.  We’re sending more email, bigger email, and keeping it longer.

Email server products such as Microsoft Exchange Server have responded to this growth in storage needs with support for more processing power, more efficient database schemas, and improved performance on storage hardware.

In fact, most of the storage performance gains of the last 4 years have been in the efficiency of the Exchange Server product itself, not in the performance capabilities of storage hardware.  Hard disks are getting bigger, but they aren’t getting faster.

As we become more reliant in the ability to retain and access email data quickly it is no surprise that we are storing more and more of it in our mailboxes.  This increase in email storage reveals some new bottlenecks in IT systems – the ability to adequately back the data up.

Backup Challenges

Backups are experiencing similar growing pains to disk storage.  Tape speeds and capacities increase through new generations of the technologies, but when disk speeds and network speeds don’t increase with them there is only so much throughput that you can achieve.  Eventually many larger enterprises reach a stage in which a nightly, full backup of the Exchange system is not possible within the backup window.

Three key technologies have surfaced to help enterprises manage these growth issues with email storage:

• Archiving
• Synthetic Backups
• De-Duplication

Archiving

Email archiving usually involves moving older, less frequently accessed data from the primary storage to a secondary storage system.  The secondary storage system may be built in to the email server, such as Exchange Server 2010’s archiving feature, or it might come in the form of a third party product that integrates with Exchange.

The benefit of archiving in reducing backup load is that once the data is stored in the archive it can be subject to different backup schedules than primary email storage.  While daily full backups of the primary storage might be a requirement, the archive stores may only require weekly or even monthly backups depending on the archive policies in place.

Meanwhile, good email archive systems still provide fast access to archived email items when required by end users.

Synthetic Backups

A synthetic backup combines the efficiency of an incremental backup (in which only data that has changed since the last backup is backed up) with the restorability of a full backup, by combining data from the incremental backup with existing data in the backup system from earlier backups to form a new, full backup.

In other words, if a file is already stored in the backup system and hasn’t changed, the backup system doesn’t need to copy it from the server again, it simply uses its existing local copy to “stitch together” a complete backup of the server.  Because not all data on a server is likely to change every day, the backup takes far less time than a full backup would, but achieves the same end result.

These synthetic backups can then be duplicated on to removable storage such as tape media to send offsite for longer term storage.

De-Duplication

Data de-duplication for backups means that multiple copies of the same data are not required to be backed up individually.  This is particularly effective in email systems, for example when 100 people all received the same email attachment only one copy of that email attachment needs to be backed up.

This reduces the amount of backup storage needed, but also the amount of backup traffic generated.  When the de-duplication occurs at the backup client itself there is less data required to be transmitted to the backup server, reducing overall backup times, yet still achieving a full backup.

Thanks to these three technologies of archiving, synthetic backups, and de-duplication the growing email storage needs of enterprises can be delivered while still achieving a reliable and effective backup regime.

3 Technologies for Improving Backup Efficiency for Growing Exchange Environments

Free ebook download: Top 10 Most Popular Troubleshooting Posts for Email Administrators

Started in 2005, the Electronic Discovery Reference Model (EDRM) Project was created to address the lack of standards and guidelines in the electronic discovery market.  EDRM is a great reference tool to develop guidelines and standards for ediscovery consumers and service providers.  EDRM helps reduce the cost, time and manual work associated with ediscovery.

Referencing the  accompanying EDRM diagram on their web site, the 8 areas lay out a structured foundation for facilitating the implementation of an archiving software solution.  This makes life easier with providing all the players standard guidelines, as part of the archiving and information retrieval process related to legal and government requests.

We will cover a cursory overview of EDRM.

Information Management
Getting your electronic house in order to mitigate risk and expenses should electronic discovery become an issue. This covers the initial creation of electronically stored information all the way through its final disposition.

Identification
This refers to the process of learning the location of all data which a company has a duty to preserve and potentially disclose in an upcoming  legal proceeding.

Preservation
Preservation for electronic discovery has become a complicated, multi-faceted, steadily-changing concept in recent years.  Certain suggested standards and guidelines have been emerging to provide checklists for those preparing to respond to electronic requests for production.

Collection
The acquisition of electronic information, which is  tagged as potentially relevant in the identification phase.

Processing
Electronic discovery processing must accommodate a wide variety of unstructured data, handle each form in a manner appropriate to its file type, and generate output that is structured in accordance with review requirements that often vary from one law firm to the next.

Review
At its most basic level the document review is used to sort out documents the company will actually provide and privileged documents that will be withheld.

Analysis
During this process, important knowledge for a case can be discerned from the large body of collected documents and email messages.

Production
With the unprecedented increase in the amount of electronic data that is being created and stored in the corporate environment, there has been a corresponding increase in focus on how that data that has been collected and reviewed is ultimately produced in civil litigation and regulatory investigation

Presentation
Displaying electronic information in front of audiences (i.e. depositions, hearings, trials, etc.), especially in native or near native file formats.

For more details on EDRM visit the Electronic Discovery Reference Model Project web site.

EDRM Guides Archive Strategy

Free ebook download: Top 10 Most Popular Troubleshooting Posts for Email Administrators

Recently I had an opportunity to meet with an associate and email administrator in New York City named Lisa Bruno.  She and her team are involved in a Lotus Notes-to-Microsoft Exchange migration.  Lisa shared some interesting insights.  Having recently migrated to Exchange 2007, one of the many questions we find is ‘what to use for backup?’.  Within Exchange you can set a “Local Continuous Replication”.   It takes an exact replica of the storage group.  Now the question is, since Local Continuous Replication is set do we need third party backup software?   In response to that question, we determined that yes we still need a backup solution.  By no means should LCR be considered as the only form of backup.  LCR also compliments any implemented archiving solutions by adding an extra utility for maintaining data integrity. Data integrity is crucial to just that one instance when archived data must be retrieved due to a legal inquiry. While it does give that option and helps in recovering quickly, it should not be the end of all.

Now the “LCR” is more of another option if a disaster occurs.  As a small business this gives us a quick reliable way of recovery.   Should a database get corrupted it automatically gives you an option to switch to the LCR copy.  To be exact in terms, it switches to a Passive copy of the database.  This minimizes the downtime.  Now how does this happen?  It all takes place by creating a log file.  This log file is called a Transactional Log.  Each time a change is made on the database it writes to the log instead of directly to the database.  This is how it determines how to quickly recover the database.  As part of enabling the Local Continuous Replication from the Exchange Management Console under the Server Configuration Node under Mailbox.  That is part of another documentation.  To recover from a corrupted database you have the option to run Restore-StorageGroupCopy CMDlet.  This will replace the LCR copy (passive copy) to become the active copy.    The exact command is Restore-StorageGroupCopy -Identity “First Storage Group” -ReplaceLocations:$true

Once the passive copy becomes the active one, LCR is disabled.  You have to enable Local Continuous Replication for that storage group.  By using this method the path of the group also changes.   Microsoft does not recommend this method but instead recommends the Restore-StorageGroupCopy CMDlet without the ReplaceLocations parameter.  This is similar to the other method but it does not change the path of the storage group.  It gives you more control to replace the database to its current path or set another path.  For either method used you will probably need to restart the server.

You can suspend and resume the Local Continuous Replication at any time.  To do so you can do it via the Exchange Management Console.  Reasoning for suspending the LCR could be for any specific reason.  But one in particular is to do an integrity check.  This you should do once in a while.  Once you are ready to enable the Local Continuous replication you can do so using the same Exchange Management Console.  Just click on Resume Local Continuous Replication.  Check the status field until it says Healthy.

Every once in a while you should check the integrity of the passive storage group copy to make sure that the database and the log file are not corrupted.  This can be accomplished by running Exchange Server Database Utilities (Eseutil.exe).

Microsoft Exchange LCR Compliments Archiving

Free ebook download: Top 10 Most Popular Troubleshooting Posts for Email Administrators