Hidden Treasure in your Calendar Data

We at Sumatra revel in our reputation as quantitative calendar geeks. One of the means we use to show our prowess with calendar data comes from some of the statistics we can read so readily from Meeting Maker databases, of which the following data we ran on February 8, 2008 is a sanitized example.

Some of the most interesting statistics, the number of meetings that are current is only 2.53% of the total meeting volume. Activities track almost exactly the same at 2.56%. As some of you have heard us say: Our highly advanced mathematics has proven that most events have already happened.

You can expect the same statistics from your Exchange calendar data on an ongoing basis.

Overall Stats

Item	Value	% Total
8Feb_DB_v8.8.0.7.mdb File size:	562,237,440.
Total number of meetings:	62,560.
Meeting timeframe:	1/1/1940 5:00:00 AM-12/31/2039 12:30:00 PM
Total number of meetings newer than 2/8/2008:	1,591.	2.54%
Total number of meetings older than 2/8/2008:	60,969.	97.45%
Total number of meetings without any mapped users:	62,560.	100.00%
Total number of meetings without any mapped users and newer than 2/8/2008:	1,591.	2.54%
Total number of Guests:	211468
Total number of guests attending meetings newer than 2/8/2008:	7,791.	3.68%
Total number of guests attending meetings older than 2/8/2008:	203,677.	96.31%
Total number of guests attending meetings without any mapped users:	209,603.	99.11%
Total number of guests attending meetings without any mapped users and newer than 2/8/2008:	7,791.	3.68%
Number of foreign guests	0	0.00%
Total number of activities:	1,869,557.
Activity timeframe:	1/1/1940-1/2/2040 8:00:00 AM
Total number of activities newer than 2/8/2008:	48,026.	2.56%
Total number of activities older than 2/8/2008:	1,821,531.	97.43%
Total number of Activities without any mapped users:	1,869,557.	100.00%
Total number of Activities without any mapped users and newer than 2/8/2008:	48,026.	2.56%
Overall Timeframe:	1/1/1940-1/2/2040 8:00:00 AM
Number of days:	36526

When we turn our attention to the top meeting users, we see some other interesting results. From the same server with approximately 2000 users, the top ten users account for almost 20% of the total meeting activity, account for about 15% of all guests, and about 3% of all activities. While your mileage may vary, it will not vary by much.

Top Meeting Users

User	Num Mtgs	% Total Mtgs	Num Guests	% Total Guests	Num Actvy	% Total Actvy	Grand Total	% Grand Total
AV Student Staff	3,025.	4.83%	5,939.	2.80%	563.	0.03%	3,588.	0.18%
Bela Bartok	1,488.	2.37%	5,514.	2.60%	3,134.	0.16%	4,622.	0.23%
Adam Ant	1,281.	2.04%	4,516.	2.13%	2,799.	0.14%	4,080.	0.21%
Claudette Colbert	1,084.	1.73%	3,318.	1.56%	12,446.	0.66%	13,530.	0.70%
Deanna Durbin	1,065.	1.70%	2,385.	1.12%	2,045.	0.10%	3,110.	0.16%
Erik Estrada	1,031.	1.64%	2,755.	1.30%	4,655.	0.24%	5,686.	0.29%
Felix Frankfurter	906.	1.44%	2,029.	0.95%	10,234.	0.54%	11,140.	0.57%
Research Team Conf Room	895.	1.43%	2,596.	1.22%	289.	0.01%	1,184.	0.06%
Gary Gilmore	857.	1.36%	1,845.	0.87%	3,590.	0.19%	4,447.	0.23%
Harry Hope	755.	1.20%	2,375.	1.12%	2,947.	0.15%	3,702.	0.19%
Top User Total	12,387.	19.80%	33,272.	15.73%	42,702.	2.28%	55,089.	2.85%
Grand Total	62,560.	-	211,468.	-	1,869,557.	-	1,932,117.	-

Now let's take a look at one of everyone's favorite subjects: Meetings and Recurring Meetings. As you can see, over 90% of meeting objects are one-time meetings. Of the recurring meetings, the most common are "Daily_Every_N" (which when N=7 means a weekly meeting).

Meeting Frequency Profile

Day	Num Recurring Meetings	% Total Meetings	% Total Recurring Meetings
ONCE	58,824.	94.02%	-
DAILY_EVERY_N	2,991.	4.78%	80.05%
DAILY_DAYS_OF_WEEK	364.	0.58%	9.74%
WEEKLY	315.	0.50%	8.43%
MONTHLY	66.	0.10%	1.76%
Grand Total	62560	-	5.97%

Recurring Meeting Profile

Day	Num Recurring Meetings	% TotalMeetings	% Total RecurringMeetings
Finite end date	3,568.	5.70%	95.50%
Ongoing (no end date)	168.	0.26%	4.49%
Grand Total	3736	5.97%	-

Trust Me (Part Four) - I see a Resource Forest thru the trees

What's the old expression -- you can't see the forest for the trees? Well, on my fourth attempt, I can.

To recap: What was the problem? I wanted to insert calendar data into Exchanged 2007 that was configured using a Exchange Resource Forest topology. The user mailboxes were linked to a accounts in a user forest. Those user mailbox accounts were disabled. You can not use Exchange Web Services to impersonate a disabled account.

What worked? Here is what I did:

• Upgraded Exchange 2007 to SP1,
• Created a service account in the resource forest
• Gave that service account DELEGATE permissions to ALL of the disabled accounts.
• Granted permission for the service account to see the user forest's AD
• Inserted calendar data!

Finding the disabled accounts, and assigning delegate rights to the service account turned out to be easy, thanks to Jian Li’s MSExchange Team Blog “How to access multiple resource mailboxes in Exchange Web Services (EWS)”. He described how it works, and (even better) created a script that has worked well for us (get the Microsoft Script.)

Oh, for the last few days the Exchange System Attendent decided to automatically start, and remain running. No idea why.

-R

Trust Me (Part Three)

Ran into two issues when configuring the resource forest and the user forest: service accounts and Address List System Service failing.

Service Account: I created two service accounts, one on each machine. The service accounts had the same name. When I went to link the resource account to the account in the user forest, accessing the linked domain controller failed.

Lesson learned - The link failed because I didn't qualify the service account with the user forest (e.g. userforest\mySvcAccount). So Exchange tried to link to the user forest domain controller with the resource service account and failed. Had I created service accounts with two different names, this problem would not have tripped me up.

Address List:

After I fixed the service account issue, the last step of in the "linked mailbox" wizard failed with an invalid address. After much head-scratching, I discovered the system attendent had failed with an MSExchangeSA event ID 1005: "Unexpected error The Local Security Authority cannot be contacted ID no: 80090304 Microsoft Exchange System Attendant occurred. "

Dave Goldman blogged about this: Creating a new mailbox in Exchange 2007 with the new-mailbox cmdlet fails with Address List Service not Available, but only addressed one aspect. My problem was choice (4) - SA stopped. I haven't figured out why it's failing, but restarting the SA allowed me to complete the mailbox link.

OK--everything is set. Next time I'll report on the results of impersonation.

Export-Mailbox -- "Exmerge" for Exchange 2007

I had to generate a five PSTs from client data inserted in our lab to help a prospect prove to his management team that the Sumatra Migration tool could read and insert their calendar data into Exchange 2007. It gave me an opportunity to test a new commandlet "export-mailbox" released with Exchange 2007 SP1. Export-Mailbox offers many of the features most of us loved in Exchange 2003's ExMerge.

Be forwarned - the commandlet comes with contraints:

• It must be run from a 32 bit client machine with Exchange Management Tools (Version Exchange 2007 SP1 or later) installed and Windows PowerShell (Download: Microsoft Exchange Server 2007 Management Tools (32-Bit));
• It requires Outlook 2003/Outlook 2007 be installed on the 32-bit client machine;
• Must be "run-as" using an account with Exchange Organization Admin or an Exchange Server Admin privs.

Here is how I exported a single mailbox to a PST:
Export-Mailbox –Identity < mailboxuser > -PSTFolderPath < pathToSavePST >

Here is how we told the client to Import the data from a PST into a client mailbox:
Import-Mailbox -Identity < mailboxuser > -PSTFolderPath < pathToSavePST >

I covered two of the other command-line variations. Ricardo Guerro's MsExchange Team blog post How to Export and Import mailboxes to PST files in Exchange 2007 SP1 details this commandlet. It comes with descriptions and screen-shots.

Yea! Another calendar migration tool for Exchange 2007.
-R

Trust Me (Part Two)

This is the second installment on my experience trying to setup a resource forest trust.

Last night I re-installed VMware's Server product on an Windows Server 2003 SP2 box that also had Exchange 2007 SP1 installed. I rebooted the box and left. I lost today's popularity contest to "Smiling Bob" after our early-morning users received "Service Unavailable" when they tried to visit our intranet, OWA, and anything that used Exchange Web Services.

What happened? The Application Event Logs show W3SVC-WP Event IDs 2268 and 2214, and System Events IDs 1002, 1039. It looks like VMware Server and Exchange 2007 are both trying to control IIS (VMware wins....)

To restore the server, I uninstalled VMware Server and reset IIS to run in native mode using this AdsUtil command line:

cscript %systemdrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32BitAppOnWin64 0

BTW, if you want to re-enable 32 bit apps (WOW64 Mode), run this command:
cscript %systemdrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32BitAppOnWin64 1

It was suggested that VMware's WORKSTATION product would be a better choice. I just installed the evaluation version. 'Sure n b'gora' ...web services and the intranet are still available. (Thanks, Chrisopher Q., for the tip!)

Now it's back to trying to build a one-way resource forest-trust.
-R

Zimbra to Exchange Migration - First Request

It happened sooner than we thought it would: we've got our first request to migrate calendars from Zimbra into Exchange. Any other interest out there in this path?

Trust me

We are testing Exchange 2007 in our 64-bit Windows Server 2003 Exchange Resource Forest Topology using two virtual servers (with VMWare -- Microsoft Virtual Server does not work in a 64-bit environment). We created two VMs -- the "user forest" and the "Exchange forest". We set up a one-way trust between the two forests, and tried to create a "Linked" user mailbox.

After a head-banging week, we can NOT get this to work. No matter what we do, when we try to link to the mailbox we get a Event ID 2130 (MsExchangeADAccess....Exchange Active Directory Provider could not find an available domain controller). We can ping the domain, the nsLookup is fine, dcdiag /v, netdiag /v -- everything comes back without errors.

Is anybody else having this problem? How did you solve it?

One of Sumatra's clients suggested this might work if we created five VMs - one for the DNS/DHCP, a second for the user DC, a third for the resource DC, a fourth for Exchange, and a fifth for the client. We will test his theory later this week (or this weekend).

Here is an image of the topology linked from the Microsoft Exchange Resource Forest Topology article:


-R

Sumatra Migration Documentation version 9.4

Based on some feedback from the folks at the University of Pennsylvania we've revised the migration documentation set.

It's now at version 9.4.

Those of you in active migrations should have it in your ftp accounts if we haven't already emailed it to you.

Since there are a fair number of folks on what we sometimes refer to as the "ninety-nine year plan" contact us if you want the most recent version.

Client Side Migration to Exchange from Oracle Calendar

I might have mentioned sometimes that things in our office go in weird phases. December was "Pennsylvania" month because of the four migrations we had going on in that state all at once.

February seems to be "can you tell us how inexpensively we can do this, preferably without you?" Sort of the exact opposite of Valentine's Day, and not really geared to get on our good side, but it does illustrate a point.

Given that we're about moving calendars server-side and avoiding the limitations I end this with, and we're really up-front about what we can do that nobody else can, we're going to give you the safe migration advice on how to do this client-side, user-by-user, and not involve us.

As is traditional for us we use John Lennon as an OCS 9.0.4 user, migrating to Outlook.


Export Your Oracle Calendar Data

From OCS, go to the File menu, choose Export Data...


Select vCalendar file:
If you select iCalendar, you can't move tasks.

Continue and make your selections:


To pull task and calendar data into Outlook you run the same process in reverse on the Outlook side.

Here we use Outlook 2003 as the example (lots of you are still using it).


Import Your Oracle Calendar Data into Outlook
Open Outlook
On the File menu, choose Import and Export...


Select Import an iCalendar or vCalendar file (.vcs):

Browse to your export file from OCS.

John's end result looks something like this:

The limitations of this process:

• It is client-side only. Each user or someone else must do this.
• It does not handle resources well. Yes, you can bring the resource calendar over, but they are not linked to the end users who are scheduled for the resource
• Recurring appointments become individual appointments.
• Daily Notes and Day Events do not automatically become "All day" events
• Guest lists are dropped and meetings therefore all become appointments

Advantages of this method:

• It is inexpensive
• It uses only off-the-shelf functionality
• With a good help desk it is probably do-able

You can go through a similar process for importing into Zimbra with pretty much the same limitations. I'll write that up if I get any requests.

Zimbra to Yahoo to (possibly) Microsoft?

You've got to start asking yourself what would Microsoft do with an open source alternative to Exchange if they consummate a union with Yahoo.

I envision an episode of Oz or some "reeducation camps" inspired by the Chinese Cultural Revolution.

Whatever comes we're still cranking away on our migration technology -- ironically we've (almost) completed the first prototype for an Exchange to Zimbra migration.

Now, of course, we're also working on migrating Zimbra to Exchange.

This is not a surprise to those of you who have dealt with us in the past. Our decision process on which projects to work on is based on simple free market economics: who is asking us to do things and are they willing to pay for it?