MDaemon Mail to Exchange 2007 via Microsoft Transporter Suite

Gentle reader,

Today we will use Microsoft Transporter Suite for migrating email from MDaemon to Exchange 2007.

Why?  Obviously because we're going to get to moving calendars for this legacy product, but you only get to calendars if you're also doing email and we get asked about email, so we're documenting it here.

This is for on-premises Exchange migration to Exchange 2007 ONLY.  You want to go into Office 365, the read our previous article How to Migrate Oracle Beehive Email into Office 365 because the method will be the same.  

Why 2007?  Believe it or not Exchange 2007 is still out there, and we still get questions on it.  Might as well deal with reality.

We've also tested email migrations with imapsync (spoiler alert: imapsync is MUCH BETTER).

But it starts with Transporter since it's there, it's free, it's from Microsoft.  What is not to love? Quite a few things, actually, but free is a big draw.

First, download and install it.  But: WHERE should you download it and install it?

You won't even be able to get to the documentation (all in the form of help files) until after you install it -- so I'll cut to the chase:  you must have Exchange Recipient Admin rights and Exchange Impersonation rights on a computer with the Client Access server role installed for Exchange 2007. Later version of Exchange will of course require slightly different methods of setting these permissions (and we've blogged on those enough).

Refer to this section of the in-application help.

For MDaemon we're going to be using the Transporter for Internet Mail. The installer seems to already know we do not have Lotus Notes installed,

but  then very helpfully reminds us in case that was what we were really after.  

We're not in this case, so let's just accept the EULA and get this done.

Now we can actually execute the Transporter

You are now in a position to actually start defining which user mailboxes you will move.

But you need to give it a list in a CSV-- and you notice a distinct lack of a user manual. What should the format of this list be?

In one of the few times I will ever write that the Help Files are actually.... HELPful, I am bidding you to Click on Help and read them.

The format is then readily copied and uploaded.

My file looks something like this:

You'll see something like this and are ready to import.

Now migrate your email.

Corrupt MDaemon meetings in a migration

We found (in the wild) conditions where MDaemon meetings were corrupt, in this case missing their Organizers or Planners.

What to do?  They're corrupt and may or may not display in an MDaemon client -- but they are in the data.

Under our usual rubric of "it is better to recover everything we can and let the user delete if they do not want it -- because it is harder to re-create something not there than delete something that is" we recover the meeting and tag it with the category "Meetings_MissingOrganizer"

So you could search for those in Outlook post-migration and make decisions about them.

MDaemon Mail to Exchange via imapsync

Gentle reader of migration mindedness,

Today we will use imapsync for migrating email from MDaemon to Exchange.

Why?  Obviously because we're going to get to moving calendars for this legacy product, but you only get to calendars if you're also doing email and we get asked about email, so we're documenting it here.

Using imapsync will cost you all of 50 Euros.

Goodness, gracious it is worth it!  

This application has significant advantages over other products:  

1. It is really simple to install and use.
2. The "sync" in the title is serious.  You can upload all the data from a user set during working hours and then cut over the incremental changes starting on a Friday after closing time.
3. You can also use imapsync on a Linux environment as well as a Windows environment.
4. It is all in your control as opposed to run through someone else's data center or through a major integrator looking to run up hours.
5. It is very reasonably priced with the most liberal license I have seen.

We use the Windows version in our testing.

imapsync runs exclusively in the Command Prompt.

First off, please make sure you have enabled IMAP on your Exchange 2013 server.

Migrating from MDaemon to Office 365 looks something like this (if you use individual passwords for users).

imapsync.exe 
--host1 147.1.41.1 --user1 zyg@sumatra.local --password1  "XXXX" 
--host2 outlook.office365.com --user2 zyg@sumatra.onmicrosoft.com --password2 "XXXXX"  
--ssl2

If you set up a service account with FullAccess, you can accomplish a migration with a command like this:


imapsync.exe 
--host1 147.1.41.1  
--user1 jimi.hendrix@sumatra.local --password1  "XXXX" 
--host2 outlook.office365.com --port2 993 --sep2 / 
--user2 jimi.hendrix@sumatra.onmicrosoft.com  
--authuser2 riuliano@sumatra.onmicrosoft.com 
--password2 "XXXXX"  --ssl2

Note in  the above, for an Office 365 target system we need to use the "--sep2 /" command. 

Executing will give you some excellent statistics and feedback.

Other things to be aware of

Exchange Configuration Requirements:

Before you can run imapsync, you will have configure Exchange for IMAP.  This requires two steps.

Step 1: Start two IMAP4 services (and configure those services to automatically start if you wish.)

By default, in Exchange 2013 the IMAP4 service(s) are stopped:

To start those services: On the computer running the Client Access server role:

1. Set the IMAP4 service to automatically start:

     Set-service msExchangeIMAP4 -startuptype automatic

2. Start the Microsoft Exchange IMAP4 service.

     Start-service msExchangeIMAP4

On the computer running the Mailbox server role:

1. Set the Microsoft Exchange IMAP4 Backend service to start automatically.

     Set-service msExchangeIMAP4BE -startuptype automatic

2. Start the Microsoft Exchange IMAP4 Backend service.

             Start-service msExchangeIMAP4BE

In our case, the CAS and Mailbox server roles are on the same box:

Step 2: Configure Exchange IMAP4 External Connection so users can see (and thus use) the IMAP server settings

Use the powershell SET-IMAPSettings cmdlet, e.g.:

     Set-ImapSettings -ExternalConnectionSetting {:993:SSL}.

This requires you restart IIS.

This is true even if you are working within the firewall  Thus in our case, the External Connection is the same as the InternalConnection.

Finally,  Verify things are working, using OWA’s Options select Account, then pick the “Settings for POP or IMAP access” link.

For more information see: http://technet.microsoft.com/en-us/library/bb124489(v=exchg.150).aspx

These sample scripts for major migrations / multiple users will help you out a lot.

http://imapsync.lamiral.info/examples/sync_loop_windows.bat

http://imapsync.lamiral.info/examples/file.txt

I really like the way this user lays out a basic sequencing for migrations including how and when to change your MX records.

http://hadjigeorgiou.com/blog/mdaemon-to-microsoft-exchange-migration/

And on the more than crucial need for advance planning and testing, please read this thread.

#MDaemon Calendar Migration to #MSExchange / #Office365: Video and Documentation

We have calendar migration from MDaemon into Exchange working.

Here's the video.

The migration features are pretty much all dealt with in the video.

The main screen of the application gives you our usual set of options:

And the configuration holds no surprises.

The site we're doing this for has decided it's more important to migrate in a phased sequence of groups of users, so we're taking the calendars over as "flat" but with information on attendees and status in the Agenda/Notes body.  If there's demand for a full-state method we can do it.  But this keeps costs down.

Mapping Users

If you have a user in your legacy domain named "zyg" and in Exchange the ID is "zyg-furmaniuk", you enter the following line in the exceptions.txt folder:

When you validate, you will find the mapped (i.e., correct for the target) address.

Note that in the above example Room.222 did not validate, so either our mapping is wrong, or the account on Office 365 is not set up properly.  

Permissions

You will need a service account on the Exchange side.  This account needs permission to write to all accounts you're mapping into.  We've already blogged on permissions a lot. 

Firewall Requirements 

1. Your fire wall must be configured to allow Time Sync (we need Port 13 and we cycle among time servers, the top two being 64.90.182.55 and 206.246.118.250).
2. Your computer time should not differ from the US Navy Atomic Clock time by more than 1 hour or mCalReader will inform you.

Why do we require this?  Short answer: Digital Rights Management and version control.

Determining your EWS URL
For on-premises Exchange, the EWS URL formula is HTTPS://CAS_server/EWS/Exchange.asmx
In ON-PREMISES you will usually have your IIS set for Windows Authentication (see http://technet.microsoft.com/en-us/library/gg247612.aspx for more details). This is also the default in hosted Exchange. Should you need to change this you may do so in the _Config_XML file by changing the HTTPAuthType parameter (options are Basic, Negotiate, ntlm, and Kerberos)
NB: You hear us talking about Exchange being a moving target in a migration. That’s true here. The default is Negotiate in Exchange 2013, and Basic in Exchange 2007 and 2010.

For Office 365 it is: 
https://outlook.office365.com/EWS/Exchange.asmx

Sumatra and MEC March 30-April 2

Two folks have asked us about us being at MEC  #IamMEC.  One's wanting calendar migration (and at over five thousand users we can talk).  The other about a private label for the holiday cmdlet.

In general we do not do conferences, but we think we're going to head to this one.   Contact us if you want to get together for a beer.  Like everyone else in Exchange programming we certainly owe Glen Scales one.

Useful Guided Walkthroughs for Calendaring in Office 365

Since one of our most popular posts ever is Shared Calendars in Exchange 2007 sp1, largely to show folks how to create shared calendars, I'm really happy that Microsoft is finally doing its own tutorials on some things that are just darned hard in Exchange.  Well, in this case Office 365.

Check out the new Guided Walkthroughs Microsoft has put up.

Of special interest to those followers of our blog are:

• Sharing calendar and contacts in Office 365

and

• Creating and managing resource mailboxes (conference rooms/equipment) in Office 365

Of course the other walkthroughs on email and mailbox access will be useful to you as well.

MDaemon Calendar Migration to Exchange

We got a wild request for migrating calendar data from MDaemon to Exchange and, seeing as how we're always looking for both an interesting challenge and a profitable business extension we brought up a server and created our traditional users and conference rooms.

Short answer: oh yeah, we can migrate the calendar, tasks, and contacts into Exchange. There are a few obvious issues and some amount of work to be done but there is no doubt about it.

Now the hard part: Is there anyone else out there who wants this?  

We're not sure that this is used outside of small and medium businesses, and they are not our traditional client base.

Opting out of Anyone on Google+ Sending you Email

Google has decided it does not have enough exploitable data traffic already and has made it a default for ANYONE on Google+ to send you email.

If you do not want this to happen to you (or your users) the Wall Street Journal published an easy-to-follow tutorial about changing this in your Settings.

Go here.

My two cents:  Google’s really pushing to be relevant in social networking.  But let us face it, this is a stretch.  They changed the whole YouTube comment system to Google+, and even the folks I know on YouTube all the time don’t give a damn.

Exchange Database Size Growth Post Undo -- LitigationHoldEnabled

Customers ask us how much space will Exchange require post-migration if they have to rollback (aka our UNDO.)  We typically suggest it's somewhere around 4x the size of the interim Access database,  and double the size of the Oracle Beehive tablespace.  This has been an inexact science, particularly since email continues to flow in (and out) of the organization during the UNDO process. 


One contributing factor to the space bloat is the "LitigationHoldEnabled" feature:  post-undo Exchange continues to hold onto deleted items (as it should!) Microsoft's Exchange team released an article and script that shows how to calculate the database Exchange 2010 and 2013 Database Growth Reporting Script.  It is certainly not needed for during testing --  you can always delete and re-create the exchange stores.  Rather, it's something to bookmark just in case you need to figure out why your DB storage requirements jump significantly.

Outlook.com makes it easy to switch from Gmail

So Microsoft is talking about how Outlook.com makes it even easier to switch from Gmail.

Their statistics on email dissatisfaction are interesting if not completely compelling.  In general we've had little experience of users at the enterprise level who are really happy with Google as an enterprise solution.

As nearly as we can tell the migration Microsoft offers is exclusively email.

If there's any corporate interest in a full-state calendar migration from Google to Exchange -- let us know.  One of the only reasons we did a migration INTO Google was with the expectation we'd have bigger opportunities getting people OUT of Google and into Exchange.

Open Source Server-Side Lite Holiday cmdlet for Exchange

Holidays in Exchange.

The words send shivers down my spine.

We got our first request for a tool to insert holidays server-side after we did a migration for the folks at HMS (shout out to you guys!) for Exchange 2003.  That was CDO-based tech, with its positives and negatives.

After Exchange 2007 came out we re-wrote the tool for Exchange Web Services  (EWS), added a variety of features and found that folks really wanted it, but not at enterprise licensing levels.  Go figure.

But it's a good way of showing what we can do with calendars in migrations, so we've produced an open-sourced lite version of a holiday cmdlet.

Sample code where we build the holiday looks like this:

So you can find the CodePlex project at sumatra.codeplex.com.

We've also hard-coded a few things (like, it's picking up your credentials to authenticate against Exchange, so you ARE the service account!).

We think of this more as a mini-tutorial on how to do a calendar-centric cmdlet.

Why a lite version and not the full thing?

Mainly because the full version has a lot of expertise behind it that we're reticent to just give to those of you toiling out there on software knock-offs.  

Where from here?

We are market-driven.  We're keeping an eye on what we hear back.

It's about time ... Outlook 2013 7 Day Preview is back!

It's about time (a calendaring geek's favorite expression) Microsoft fixed this issue!

I was very pissed that Outlook 2013 RTM no longer showed "future" appointments in the ToDo bar. I was thrilled to learn it's available -- see Microsoft's KB# 2837618. 

And now for the really good news:  The patch causes Outlook 2013's IMAP folders to fail to synchronize.  I'm thrilled to see my appointments 7 days out, but even more pissed that I can't get new mail sent to my inbox, or preventing me from setting up OOF messages. 

InfoWorld said it better than I could in a recent article: Botched Outlook 2013 patches KB 2837618 and KB 2837643 break Out Of Office reply, Free/Busy, and more. InfoWorld found a detailed synopsis of the issue, written by Matthew Stublefield from Missouri State University.

Bottom line: DO NOT apply this patch until Microsoft figures out how to fix the fiasco.

Oracle Beehive Unexpire and Unlock Accounts

Just to record all of this stuff in the event you need it.

If your passwords in Beehive / Oracle have expired, then Oracle Unexpire and Unlock Accounts is really, really useful.

So too is Oracle Database Security Checklist.

Protecting customer data from government snooping

Hell freezing over is too cliche.  

But Microsoft's Blog Entry from their General Counsel  is that kind of amazing just because I find myself in the rare position of applauding Redmond for taking some stand for user privacy against the knee-jerk, butt-covering, bureaucrats who think Orwell proposed a really great idea.

Keep it up, guys!

Yes, this is one of our rare posts that has nothing to do with calendaring but it is so momentous I need to get on record as supporting it.

Outlook 2013 Security Update breaks Out of Office for Exchange 2007 Mailboxes

Just what the title says:  Outlook 2013 Security Update breaks Out of Office in Exchange 2007 Mailboxes.  The link tells you what to do about it.

BlackBerry Open Letter

I just get back from my second trip to the airport today and what's in my inbox but a weird BlackBerry email.

In similar words from Hamlet: methinks they doth protest too much.

Here it is with brief commentary on BerryReview.

Maybe BlackBerry is not living under a rock for acknowledging they are under siege, but they do not acknowledge that this is not in the same envionment as before.  Like Thursday's Macy's Parade the balloons have collapsed and the parade has gone by.  Rather than a leader of the smartphone marathon BlackBerry is now in a dash to safeguard its highest hanging fruit while the lowest hanging drops in other baskets. 

We've had our love/hate affair with the BES server over the years. As both BES and our migration technology for E2K3 were CDO applications anything that screwed up BES (usually permissions or CDO calendaring bugs) were harbingers for what was going to happen with our code.  To this day some of our all-time most popular blog posts are referenced on the BlackBerry forum to deal with permissions issues.

I wish them good fortune.  

But I also wish you the same thing we advise everyone:  look at all your options and test the snot out of them with the knowledge that your user base is seeking direction from you now and your next performance review is coming faster than you think.

Hard Limit of 25,000 on Beehive Calendar Objects

While stress testing our migration methods -- we did a major push for performance optimization aka SPEED -- before a migration this past weekend and of course that meant a few bugs crept in, we got this message from Oracle Beehive.

So if you did not know it before, there's a hard limit of 25,000 meetings in Oracle Beehive Extensions for Outlook.  Hard as it is to believe we do know of users who have exceeded this in other legacy systems.

External Contacts in an Oracle Beehive to Microsoft Exchange Calendar Migration

External email addresses in meeting invitations.

We do not make them live in migrations.  The reason is simple: they confuse the heck out of the attendees and it's not worth it.  While you can communicate to your internal folks that a migration is going on and we are taking care of re-creating the meetings, we cannot reach out of your organization and re-accept or decline for your external guests. 

But we can let YOU know about them.

So as of all forthcoming version of our process we're keeping the record of external guests in the meeting agenda, exactly as you see here post-migration.

When Impersonate Permissions Fail DURING a Migration

Interesting case came up this weekend.  

An 850 user Beehive to Exchange Migration validated fine (notable for two things: First European Beehive migration and first multiple instance Beehive migration) and then while in migration started throwing impersonate errors for what became a total of ten users.

Now ten out of 850 is about 1% -- which to us is still too great an error rate for us to tolerate.  

But it seemed really wasteful to undo the entire migration for ten users.

Here's what we did.

We modified the insertion to allow for a single user.  Not via validation of that user which is a good test procedure but it would be insufficient here.  You need the validation file to be as entire as you can get it -- otherwise there will be inaccuracies.

First fix your permissions issues with those users.

THE ABSOLUTELY SAFEST WAY

Here are the steps we recommend:

•     Create a new sub directory for each of the accounts.  Remember to copy configuration and the two account validation and mapping files
•       Launch the bcalreader UI, and set the lower and upper limit for the first account’s SMTP address root, e.g.:  “jimi.hendrix” to “jimi.hendrix”
•     If there are multiple accounts that have this root, follow the limit values with an “@” sign.  For example, if you have two accounts “jimi.hendrix”, “jimi.hendrix2”, and you only want to process “jimi.hendrix”, set the limits: “jimi.hendrix@” TO “jimi.hendrix@”           Otherwise the tool will migrate “jimi.hendrix” and “jimi.hendrix2”, and you will end up with duplicates in the “jimi.hendrix2”’s mailbox.
•     Validate, then press the “process all” button.
•     Repeat for other users

NOTES

• This inserts all meetings/contacts/tasks owned by the user, and process all invitations received from other users.
• However, it will NOT process invitations sent from this account to other accounts.  Those invitations will remain in his invitee’s in-boxes.  We suggest you leave it that way.
• Communicate to your end users that the migration may have left some meeting invitations in their inbox.  Suggest they process those invites the way they did in Beehive (e.g., if they accepted the invite in Beehive, accept the one in Outlook;  If they declined the invite in Beehive, decline it in Outlook.

THE ABSOLUTELY BEST WAY

The best way in this case means getting the outcome you could have had if the impersonate had not failed.  But it is a little more intricate and has the possibility you press the wrong buttons.

Do this in the cold light of morning.  It is way too easy to hit the wrong button when you are sleep deprived and stressed. 

• Create a sub-directory that you will use to process the accounts that had impersonation problems.
• Copy bCalReader v2.0.11 or greater and the validation file.
• Change the limit to the account.  Put an “@” after the SMTP address, e.g.: “Jimi.Hendrix@”
• Press “Process All”.
• Repeat for the remaining accounts.

Once you have added data from the impersonation-problem accounts, return to the “main instances” of bCalReader.

For each of these instances:

1. Check the “show individual steps”, and re-process the invitations as follows
2. Press the “Respond to Invitations” button on each instance.  When they are all done,
3. Press the “Apply response Exceptions” button on each instance.  When they are all done,
4. Press the “Update Tracking Info” button on each instance.  When they are all done,
5. Press the “Final Cleanup” button on each instance.  
6. When they are all done, you are finished! 

Multiple Insertion Instances in Oracle Beehive Calendar to Exchange Migration

Now that we have Beehive to Exchange migration running reliably and in test in both the USA and Europe, we're working on improving the throughput with our old friend: Multiple Instances aka Segmentation.

Since re-creating calendar state in Exchange requires sending, responding, and creating exceptions for meeting invitations, this technique does require some special handling, which we're detailing here.

First, the screen morphs a little:

As we've done in the past, this is an alphabetic list, so 

1. Make sure when you do this you cover the entire alphabet.  Once a site left out the letter "k" -- yet another great example of why we built "UNDO" capability into the process.

2. Sometimes resources or rooms start with numbers, so make your first installation "SPACE" instead of "A".  If you try to use A this comes up: 

3. Remember what we said earlier about sending invitations and responding to them?  Suppose that Adam Ant invited Lex Luthor to a meeting, but Luthor is in a different segmentation.  It's possible one instance could be trying to respond for Luthor before he gets the invitation and suddenly everybody gets confused.  No worries.  After the invitations have been sent for each instance the code pauses and tells you to wait for all the other instances to catch up.

How to use this:

• Go to each computer you want to use 
• Install bCalReader and its associated .DLLs
• Validate and map all your users on one machine and then COPY the account_validation.txt and alt_account_maps.txt to the other instances.
• Proceed.

As always, TEST TEST TEST before you go live.

Performance Optimization in a Beehive to Exchange Calendar Migration

We've been fine-tuning the performance in the Beehive Calendar to Exchange calendar migration.

A little bit of background is due here.  We use the rate of 850 objects per minute as a rough gauge of Insertion time into Exchange in our Meeting Maker or OCS migrations.  But that is ONLY for insertion into Exchange from our own data structures and does not include the time for any processing of Legacy calendar data (the Extraction phase).

For Beehive, we've combined the EXTRACTION and INSERTION into one application where we read directly from Oracle's relational database.

And needless to say performance varies hugely with the Beehive topology and servers.

So a few general results:

• Your migration performance improves  20-30% if you SHUT DOWN Beehive (technically our results were 18% to 33%, in a variety of configurations, but rounding makes them easier to wield).
• Going into on-premises Exchange yields more reliable performance than insertion into Office 365.
• Extraction involves a certain amount of overhead that is difficult to remove and highly dependent on Beehive system and configuration.  Think of it this way: it always takes a certain amount of time to pull up a user.
• Given that, look at your summary report for you top users and use that data in conjunction with your test experience to gauge your insertion times.  If you need help analyzing this contact us.
• Calendars with more objects take longer to insert so only take data as far back as you need.  If we see serious requirements for lots of past data we'll modify the code to allow you to insert current data as quickly as possible (so you are functional) and then go back and insert your historical archive.  This has been a regular feature of our other migrations for a while now, but Beehive migrations are relatively new.

Contact.Birthday property in EWS

Has anyone else noticed that the Contact.Birthday property in EWS seems to not work?  We noticed the same thing with Contact.WeddingAnniversary.

It's a small nit -- but it's kind of annoying us.

Server-side Holidays for Microsoft Exchange cmdlet

OK, we'll let you use the cmdlet for server-side holidays.

It works with Exchange 2007, 2010, and 2013, as well as Office 365.

You will have to set permissions to be able to access calendars.

You can download it here.  

Update December 10, 2013: Link removed while we consider alternates for the holiday cmdlet.
Update December 17, 2013:  We created an open-source cmdlet project on CodePlex.

Throttling in Exchange 2013

The way you control throttling changed in Exchange 2013.  If you recall, we set a different set of throttling attributes in Exchange 2010 when impersonating users.  Those attributes differ from user throttling attributes (this is a good thing for migrations!!) In Exchange 2010 there were seven throttling attributes.  In Exchange 2013, there are only TWO (again, we're focused on EWS impersonation access):

• EWSMaxConcurrency
• EWSMaxSubscriptions

To define a policy "SuThrottlingPolicy", and set it to your service account, "exsu", use the following:

 New-ThrottlingPolicy SuThrottlingPolicy -EWSMaxConcurrency $null -EWSMaxSubscriptions $null
 Set-ThrottlingPolicyAssociation -Identity exsu -ThrottlingPolicy SuThrottlingPolicy



For the inquiring minds, the following are set in Exchange 2010, but NOT used in Exchange 2013:

• EWSFastSearchTimeoutInSeconds
• EWSFindCountLimit
• EWSPercentTimeInAD
• EWSPercentTimeInCAS
• EWSPercentTimeInMailboxRPC

Here are the errors will you see in our log files if you hit the throttling limit (taken from MSDN:)

Error	Throttling policy parameter	Description
ErrorExceededConnectionCount	EWSMaxConcurrency	Indicates that there are more concurrent requests against the server than are allowed by a user's policy.
ErrorExceededSubscriptionCount	EWSMaxSubscriptions	Indicates that a user's throttling policy maximum subscription count has been exceeded.
ErrorExceededFindCountLimit	EWSFindCountLimit	Indicates that a search operation call has exceeded the total number of items that can be returned.
ErrorServerBusy	EWSPercentTimeInMailboxRPC EWSPercentTimeInCAS EWSPercentTimeInAD	Occurs when the server is busy. The BackOffMilliseconds value returned with ErrorServerBusy errors indicates to the client the amount of time it should wait until it should resubmit the request that caused the response that returned this error code.

Or, the HTTP status codes that are returned by throttling errors:

HTTP status code	Description
HTTP 503	Indicates that EWS requests are queuing with IIS. The client should delay sending additional requests until a later time.
HTTP 500	Indicates an internal server error with the ErrorServerBusy error code. This indicates that the client should delay sending additional requests until a later time. The response may contain a back off hint called BackOffMilliseconds. If present, the value of BackOffMilliseconds should be used as the duration until the client resubmits a request.
HTTP 200	Contains an EWS schema-based error response with an ErrorInternalServerError error code. An inner ErrorServerBusy error code may be present. This indicates that the client should delay sending additional requests until a later time.

Enterprise Calendar Metrics: The View from 10,000 Meters

Our last post was a start at getting at what kinds of time-based information is in Microsoft Exchange / Office 365 calendars that can help you get a hand on what is going on in your organization.

We want to draw the distinction between time management (something individuals either do or not do for their personal schedules) and calendar metrics (something you can read from the aggregated calendar data in your Microsoft Exchange server).

Here we'll start taking a look at some of the global reports you can extract that enable some insight on what your corporation is doing.

Some data we extracted a while ago generated the following results:

Not surprising (to anybody who's worked in an organization with more than fifty people) the number of meetings drops at lunchtime.  We interpret this as people wanting to get some private time, but any anthropologist out there will ask how we normalized our results, set up proper controls, and did double-blind studies.  None of us are anthropologists.  What is more interesting to you the manager are the twin peaks of mid-morning and mid-afternoon for the most popular meeting time.  Plan your resource use accordingly.

The outlying time of midnight - 1 AM we think was either security or manufacturing.

Which hour do people meet leads to the question which DAY is favored.  In the above report we found that meeting frequency peaks on Tuesday.  In LOTS of both legacy and Exchange data we have seen Wednesday and Thursday being the peak, but almost never Monday or Friday.

For the true calendar cartographer a totally nerdy but relevant question is: what kinds of recurrence patterns are established in my organization?  While recurring meetings and appointments are really neat, there data shows that the majority of meeting objects (we'll get to that in the next sentence) are one-time.  But I said objects.  So a weekly meeting counts as ONE object even if it occurs 52 times in the course of a year.

These combined with lists of the top users of calendar functionality are a good place to start to get a handle around enterprise-level metrics.

What can we do about individual resource metrics?

So glad you asked.  Hang on for the next post.

How does your enterprise spend its time?

Linkedin blogs contain interesting thought pieces.  Three recent posts caught my attention – they talk about how to cut down/eliminate unnecessary meetings:

Jeff Weiner (CEO LinkedIn) notes his managers complain about managing their inbox and their meeting schedules in his post: A Simple Rule to Eliminate Useless Meetings, and The Importance of Scheduling Nothing.

Rajat Taneja (CTO, Electronic Arts) wants to Cut Down on Unnecessary Meetings.

I found Rajat Taneja's post most interesting.  He talks about understanding "how we spend our time," and then he tracked how he spent his time.  We think the act of measurement is a critical component that will drive the CEO to force the organization to change the meeting behavior.

Both of these are geared towards the meeting organizer, that is, changing individual behavior.  But both of these being high level executives, the more interesting question they do NOT pose is: can we get a handle on how the enterprise is spending its time?  Are we being efficient or not?  And can we read this out of Microsoft Exchange?


Having been looking at scheduling metrics off and on over the years we've been trying to figure out how to interrogate the Exchange server to derive useful metrics on this for analysis on the organization / enterprise level.

It's easy to use server-side tools to get a handle and do reports on how much email traffic is generated and disk space is in use.  Email analysis has relatively little dependence or significance on job title or function.

But calendars are precisely the opposite!

As a Tech CEO you usually want your engineering staff working on engineering problems and not in any more meetings than they need to be.  But your sales people need to be meeting with clients or they're not doing their job.  In between there's a wide gulf as much dependent on individual corporate culture and goals as on anything taught in a management course at Business School.

We took a cut at that measurement, in our recent Oracle Beehive migration tool.  See "The Oracle Beehive Calendar Metrics" post for a screen shot of the metrics.  Those of you who have been through migrations with us have seen similar reports on your legacy data.

We've learned over years of analyzing data that the top ten meeting users is sometimes a surprising revelation to corporate management -- usually when they find that one of their top users is a conference room.

And this actually is where we made what we think of as our first big break through: that the inanimate objects in the corporation have a lot more importance than you would first think.  

In fact -- since the only major issue with them is "how much are they in use?" it's relatively straight-forward to do reporting on them and we've already sliced that a few different ways for people.

We'll show sample reports in a follow-on post.

More #Oracle Beehive Calendar Metrics

Our latest version of the Beehive to Exchange / Office 365 migration code includes more table of metrics of Beehive calendars.  I show some of the additional stats we generate here using only two users in our test server so it's easier to take in.