In this post, we will discuss the following topics:
- The concept of the Exchange Database including the Extensible Storage Engine or ESE
- The way transaction logs and database files work together – storage architecture
- How to create and mount databases
- Some best practice concepts and implementing them with a newly created mailbox database
It’s important for you to visually realize that every email that goes to your Mailbox server must go into a database, and this creates challenges because of the huge variety of messages Exchange handles. From the tiny on-line emails to the monster MB emails with video attachments.
The I/O profile of a Mailbox server is not predictable (RANDOM). Read/write that occurs between memory and disk is substantial. At times there may be waves of messages, other times may be idle.
Exchange uses the Extensible Storage Engine (ESE) database engine (MS has mentioned moving this to SQL but that has been determined to be too much of an investment, so ESE remains in Exchange 2010). The ESE engine has received some enhancements to improve I/O by 70% (meaning, Exchange 2010 can read/write emails to disk 70% faster than 2007 using the same engine). These improvements included increasing the page size from 8kb to 32kb, storing header data in a single DB table, and compressing attachments. In turn, because of these optimizations, you actually have more options for using lower-end disks for your Exchange server.
Disk Types for Exchange:
- Better IOPS performance mean flexibility in storage options:
- High-end storage (SANs or RAID arrays)
- Exchange 2010 uses reduced I/O and this allows for Lower cost SATA disks or Just a Bunch of Disks (JBOD) storage
- One important point to keep in mind is that Microsoft removes Single Instance Storage with Exchange 2010
- The idea behind SIS is when a message is sent to a bunch of people (perhaps with a large file included) the original message is stored once.
- SIS is replaced by database compression technology and new tools to help administrators to purge mailboxes and reduce the overall size of the database
To visualize email flow consider the following:
The email comes in, the CPU/Memory sends it to the hard disk but not immediately into the database, because it might be busy. First it goes into transaction logs, which ensure the data is written into the database. Transaction logs don’t do anything, they are 1 MB in size and the amount of files/logs grows. But they are written into the “one monolithic database” file. The exchange store uses write-ahead Transaction Logs and CheckPoint files to help prevent data loss. These files record all changes that have been committed to the in-memory database. While checkpoint files record which data has been committed. If the DB is corrupted, and the transaction logs are safe, your DB will be up to date. Its recommended to keep TL and DB on separate disks. JRS files are used when the hard drive runs out of space and help to stop the queue DB cleanly. When the store runs out of hard disk space, the transport service will be stopped. At 1 GB it will run out of space and is BAD. Space cannot be reclaimed. *Make extra effort to ensure you do not run out of disk space!*
In the Exchange Management Console -> Mailbox server role -> Database Management tab, we can see the database file path and where the Transaction Logs are located.
In the Mailbox Database folder on the hard drive, in the transaction logs, when the placeholder E00 log is complete, it will be renamed.
When you create a new Mailbox, you can change the DB file locations. (PS command: new – mailboxdatabase Server…)
Exchange 2010 Database Best Practices:
- Place transaction logs and database files on separate disks (off the system disk and/or the location where Exchange is installed – you can move the location of your DB)
- Place transaction logs on a mirrored volume
- Place database files on a RAID 5
- Use the Exchange 2010 Mailbox Server Role Requirements Calculator to help you determine your storage needs. The latest version can be found here: http://blogs.technet.com/b/exchange/archive/2009/11/09/exchange-2010-mailbox-server-role-requirements-calculator.aspx
- Note: Standard Edition supports 5 databases. Enterprise Edition supports 100 databases
For an existing DB, you can “Move Database Path” at any time and change the default locations for transaction logs and database file location.
EMC -> Organization Configuration -> Mailbox -> Database Management Tab -> Right-click on Database -> Properties ->
Maintenance Tab ->
Database Management Properties
- Mount and Dismount the Database
- Move Database and Log Paths
- Background Database Maintenance (24×7 ESE scanning)
- Circular Logging
- Storage Limits
- Deletion Settings
We will touch upon Journal Recipient later.
Here the Exchange Mailbox Database Maintenance Schedule can be adjusted or Customized.
Database maintenance performs the following maintenance tasks:
- Purging mailbox DB and PF DB indexes
- Maintaining Tombstones
- Cleans up deleted Items Dumpster
- Removes public folders that have exceeded expiration time
- Removes Deleted Public Folders which have exceeded the tombstone lifetime
- Cleans up conflicting Public Folder messages
- Updates server versions
- Checks schedule plus Free/Busy and offline folders
- Cleans up deleted mailboxes
- Checks message tables for orphaned messages
- Cleans up reliable event tables
By default, from 1am to 5 am, these tasks above are completed.
An online defragmentation process will be run to free up pages in the database at the end of every maintenance schedule. This is done to reduce the amount of I/O.
Legacy versions of Exchange would do a DB check-sum and look at every page to see if there was corruption at end of maintenance schedule.
However, ESE Scanning does the DB check sum process for corruption on the fly, outside of the maintenance schedule. Un-checking the option for ESE 24/7, the defrag will still take place at the end of the scheduled maintenance.
– Enable Circular Logging – saves disk space – allows exchange to overwrite transaction logs. If your database is corrupt, the transaction logs will restore the DB from the logs. It’s not recommended to use this option – but good if you want to save disk space (perhaps you can turn this on for specific databases)
Limits Tab:
Storage limits – mailbox and storage limits
By default, every day at 1am if the user hits these limits, they will be sent a notification that their mailbox has gotten so large that they will either be prohibited to send or send and receive.
Issue Warning at (kb):
Prohibit send at (kb):
Prohibit send and receive at (kb):
Deletion settings defaults:
Keep deleted items for (days): – deleted items do not count against user’s total mailbox size
Keep deleted mailboxes for (days)
Don’t permanently delete items until the database has been backed up (recommended to enable this option)
In summary, there is a lot to consider when managing an Exchange database. We talked about the concept of an Exchange Database, Transaction logs and database files, mounting/dismounting databases, best practices for configuring DB’s and logs, and finally, maintaining our databases with best practices.
A good majority of the content provided in my Blog’s Exchange series is derived from J. Peter Bruzzese’ excellent Train Signals Exchange Server 2010 Video Disk Series, as well as my own Exchange 2010 lab. Trainsignal.com is an invaluable source for accurate, easy to understand, IT information and training. http://www.trainsignal.com