Tips and tricks for optimizing IMAP migrations from Microsoft

Good to see this:

Tips and tricks for optimizing IMAP migrations: https://t.co/UH5MCjrVVR pic.twitter.com/MEPMya69rh

— Microsoft Exchange (@MSFTExchange) August 24, 2016

 But they left out the most important suggestion:  evaluate imapsync instead of Microsoft's black box.

How to Create Shared Calendars in Office 365 Part 2, Server-Side

Gentle Reader of Scheduling Intent,

When last we spoke it was of shared calendars / group calendars client-side in Office 365,

Today we will look at the same capability server-side.

In what a psychoanalyst would call a "breakthrough" Microsoft has established a user-interface for an Office 365 Admin to create a Group.  Log in as your Administrator and it is easy to navigate to:

from which

You will need to hit REFRESH (under "More") to see it but you can then add users / owners.

Not bad.

Can you still do this with PowerShell as in the olden days of Exchange 2007?  Sure.  Leave a comment if you'd like to see that.

Oracle calendar to Zimbra via ICS does NOT solve all your problems....

In response to our post: Oracle + Zimbra?  we got at least one person writing in saying "But we can just import the ICS files."

Yes, you can use ICS export/import and it SORT OF works.

You will not have your recurrence patterns and every recurring meeting and appointment will come in as a series of single, disconnected instances.

That is for starters.

Then there are the issues of mapping users throughout the OCS namespace so they map to the new IDs in your Zimbra namespace.

Then there's the issues of conference room / resource behavior that OCS allows but Zimbra does not and how you should deal with that.

How to Create Shared Calendars in Office 365 Part 1, Client-Side

Gentle Calendar-Using Reader,

The team at Sumatra feels it is time to update one of our most popular posts ever, Shared Calendars in Exchange 2007 sp1, for the Cloud Era of Office 365.

There are two methods for doing this:  Client-side and Server-Side.  We're going to deal with each in separate blog posts.

You might think that since we did this for Exchange 2007 in the year 2008 of the Common Era that this should be a simple matter of a few well-pointed clicks in a graphical user interface.  It HAS become easier.

Let's look at Client-Side (i.e., End User-Side) first from the perspective of a user in OWA (the process from Outlook is not exactly the same, but you'll get the idea)

First create a secondary calendar:

Now you need to name it something (In this case: "Zyg's Shared Calendar" it could just as easily be "QA schedule" or "group vacations and travel"):

Now let's Right click on the secondary calendar and select Share Calendar to pick who we want to see it and how much power we want to give them to post or edit things:

Note that we can give Russ the ability to Edit or add things and Jimi only the ability to read what's there.  This is darned useful.

From the Microsoft-generated text in Jimi's invitation it is pretty clear that Redmond expects users to have trouble with this process.

Notice that accepting the invitation automatically lists the shared calendar in Jimi's calendar list AND that Jimi can now see the events the owner has posted:

and while Jimi cannot edit or add to the events, Russ can edit any events (and note that in good social practice he is annotating that it's him doing and mods -- Exchange/Outlook/OWA do not do this for you

Note -- you need to be careful what you put into these calendars because events do NOT show up as BUSY in a free/busy check.  So it is very easy for Zyg to book a meeting at the same time.  Note that below you do NOT see a conflict in the Scheduling Assistant.

Another way of doing this is via Groups (I think of this as "SharePoint Lite"):

You get similar options:

Because of  the license requirements of nickel-and-diming Office 365, you will need both a OneDrive for Business license and an Exchange Online license:

Oracle + Zimbra?

My latest newsletter from Zimbra had the following:

As the referenced Press Release Oracle Gives Partners a Fast Path to Cloud explains: "Zimbra plus the Oracle Cloud means users can start small and grow, or start big and get bigger"

Forget the fact that Zimbra's had its own cloud that's been promising the same thing.

The Zimbra back-end of mySQL (now owned by Oracle) makes this start to look more interesting than it might otherwise seem.

Consider this:  Microsoft has the very successful Exchange and Google has its office suite.

What Oracle has is the obsolete Oracle Calendar and the moribund Oracle Beehive.  

Given that Oracle traditionally cannot stand not having anything in the enterprise that Microsoft and Google already have. are we looking at the first tentative steps towards an acquisition?

OCS / Beehive into Zimbra is doable on your own if you want, but you'll need to be really careful of resources.  If you want to do it the right way and you've at least a few thousand users drop us a line.  We've got a long history taking both Oracle and Zimbra into Exchange (I mean, check out the rest of this blog).

After 20 Years Sega Saturn DRM Cracked.... and what it has to do with calendaring.....

Bravo to Dr Abrasive for cracking the Sega Saturn DRM method.

What on earth does this have to do with calendaring?

Well, since we broke Meeting Maker, Oracle Calendar Server, Zimbra, Oracle Beehive, and a few other encoding schemes, we're not allowed to talk about, we're sort of connoisseurs of reverse engineering and call out kudos where they are warranted.

What's kind of amazing is that none of the engineers came forward in the last 20 freaking years with anything that would help this.  Of course, for Meeting Maker the group of hacks finally working on it only started to contact us after we'd done everything significant to read their data and insert it into Exchange.

And don't even get us started about the Oracle people!!!!

Troubleshoot Outlook Connectivity in Exchange 2016 on-premises

We hit connectivity issues running Outlook 2016 on Exchange 2016 at least once a week.  Troubleshooting is difficult because of many platforms and technologies in play. 


A recent blog post from the Microsoft Exchange Team titled "Checklist for troubleshooting Outlook connectivity in Exchange 2013 and 2016 (on-premises)" promises to help us figure it out.  There wasn't a check list, per se.  Rather it was an organized collection of troubleshooting tips and techniques. We hope it helps us and helps you.  (Note: this is NOT intended for Office 365 connectivity issues!)




First, here are examples of the connectivity problems they cite, and we've hit:

Clients prompting for credentials (intermittently or continuously)

Clients getting disconnected

Clients are unable to establish a connection

Clients freezing or going unresponsive

Here is a summary of their tips and recommendations:


* Ensure that everything is fully patched.  We find Office Configuration Analyzer Tool (OffCAT) quite helpful.  In fact, Microsoft Office released OffCat Version 2.2 June 2016.  Download v2.2 here.


* They recommend cached mode vs. online mode to smooth out the user experience.  We agree it helps, although it masks connectivity problems.


* Ensure CAS servers are not turning off NIC cards, use outdated drivers, or are not configured for power saving mode.  The same holds true for the load balancer -- make sure keep-alive and idle timeouts are set above the 15 minute threshold.


* Too many cores:  it's hard to believe that you can have too many cores, but you can.  Don't have any more than 24 cores per server


* Configure Exchange performance monitoring ("perfwiz").  MS points you to two articles:  Troubleshoot High CPU Utilization in Exchange 2013, and Exchange Monitoring tool, "Exmon"


* Logs: The article recommends Outlook logging, HTTP logging, IIS logging, Exchange Logging, and RPC logs.  They recommend a tool, Log Parser Studio, to help parse the logs.




Writing this blog was the easy part.  Now we'll have to try each suggestion until we discover what's causing the client connectivity problems.

Double-Booked Meeting Rooms in Office 365 (and how to avoid them)

Update: November, 2017: If you look at newer postings you'll see details on our new method of finding and fixing double booked meetings.  If you want to try it, contact us with the size of your Exchange installation and your test domain (really, please don't start in production!).


Let's talk meeting rooms (aka resources) in Office 365 / Exchange 2016.

Why might you NOT "Allow repeating meetings" when configuring meeting room booking options?

Short answer: Because you have to be really careful.  Especially in Office 365 if you are working entirely from the user interface.

Note that for a one-time meeting where I have the Room 222 booked all day, attempting in Office 365 to book that resource does not even allow it to be seen as an option (this is a REALLY good thing):

But setting up a recurring meeting that overlaps with it easily allows me to book the conference room over a conflict, which might cause me as the meeting organizer some worry down the road.

To avoid this unpleasantness entirely, you need to use Set-CalendarProcessing via PowerShell.

To see what the settings on this resource are:
Get-CalendarProcessing -Identity "Conference Room 222" | fl

to set the ones that deal with conflicts/recurring items:

Set-CalendarProcessing -Identity "Conference Room 222" -AllowConflicts $true  -ConflictPercentageAllowed  80 -AllowRecurringMeetings $true  

Note that here we allow a conflict percentage of 80, which is below the threshold we set when setting up our demonstration.  If you want to allow no conflicts at all via auto-booking: 

Set-CalendarProcessing -Identity "Conference Room 222" -AllowConflicts $false -AllowRecurringMeetings $true  

The Microsoft manual for this goes into some good detail:
https://technet.microsoft.com/en-us/library/dd335046(v=exchg.160).aspx

And as always, be on the lookout for functionality that works in Exchange on-premises but differently in Office 365.

Side note: in our classic Sumatra Utilities for Exchange 2003 we included a double-booking checker, and this would not be hard to do but we hear little demand for it in Office 365/ Exchange 2016.

Addition January 23, 2017:
Folks, this post is insanely popular.  And we're wondering why.
Our initial motivation is to make sure people do not run into this during a calendar migration.  But the number of folks reading this regularly indicates it's a common problem.
Now -- Exchange does a VERY GOOD JOB of notifying a meeting organizer of exactly WHERE conflicts are at the time of booking.  Minimal personal organization skills then allow you to make modifications to individual instances to smooth things over.
Only thing we can figure out is that folks don't do this and they're looking for an upcoming "oops!  You have a conflict in that meeting room coming up!" notice.
Is that the case or is it something else?
We never learned anything listening to ourselves talk, so if you have something to share please let us know.

Exchange 2016 Permissions Debugging Protocol

We put together a permissions debugging protocol for Exchange 2016 / Office 365 calendar migrations.

You can download it here.

Turning off reminders during a calendar migration to Microsoft Exchange / Office 365

We've been doing a bunch of Oracle Calendar Server to Exchange migrations lately and this has the benefit of giving me lots of detail oriented stuff to write about, and the downside that I'm getting tired of writing about it.

So what happens when you do not want to get reminders for meetings as you migrate into Exchange?

This is an Outlook configuration issue.  That is to say, this is normal Exchange functionality.  Meeting requests in Exchange create a tentatively accepted calendar item that carries with it the “default reminder time” set in Calendar Options.  Note that this applies to meeting invitations, not to appointments (calendar events with no attendees) which will all go in with no reminders set in an OCS to Exchange migration (because Oracle Calendar Server strips them off in the ICS export).

The PowerShell syntax, to TURN OFF for all MBX USERS (before migration)

get-mailbox -resultsize unlimited  -filter {isResource -eq $false} | Set-MailboxCalendarConfiguration -RemindersEnabled:$false 
-DefaultReminderTime:0  -ReminderSoundEnabled:$false


to TURN ON (after migration)
get-mailbox -resultsize unlimited  -filter {isResource -eq $false} | Set-MailboxCalendarConfiguration -RemindersEnabled:$true 

-DefaultReminderTime:00:15:00 -ReminderSoundEnabled:$true

This can be set in PowerShell for one user as in this screenshot:

Finally, note that if a user is logged in or connected (OWA/Outlook) setting these preferences will not kick in until the user logs out and logs back in.  So, if you are testing it, you will see the settings change BUT you won’t see an reminders turn off until you log out/in.

Microsoft SQL going after Oracle? Don't forget the calendars in Oracle Calendar Server or Beehive!

Microsoft waves the 'free licenses' flag to try to grab Oracle database users?

Sure.

I'm not certain that the arrogance of Microsoft is preferable to the arrogance of Oracle (what's the difference between Larry Ellison and god?  God doesn't think he's Larry Ellison.).  But free is a hard argument.

Just remember we can migrate your enterprise calendars (Key word:  Enterprise-- if you have 100 Oracle calendars, just use client-side methods.  If you have 500+ users and mission-critical meetings, check us out).

Special case: When there is no user GUID in your OCS Exports

The ICS export for Oracle Calendar Server is inconsistent with exporting GUIDS:  As in this edited real world example:

Why is this relevant to anything that you care about?

Because interspersed with these very valid users you wish to migrate are also decommissioned users you do not.  Our process will send invitations to them during migration.

For valid (whether mapped or not) users in Exchange this is not an issue and they will be dealt with properly.

For users who have been decommissioned and do not have addresses on Exchange, this will result in NDRs (Non-delivery reports) being send back to the meeting organizer for these decommissioned users.  We have no way of knowing who they are (since they look like other perfectly valid users) and some sites keep these users in attendee lists as a matter of historical record.

The consequences to you:

1.       Our clean-up process REMOVES any NDRs before end users see them post-migration.  We find users and admins value this ease of use and clean migration

2.       The decommissioned user will still be in the meeting attendees list.  So: a.) All attendees will still see that user in the attendee list and b.) In future meeting updates post-migration the organizer will again get NDRs and it will be up to them to remove the user at that time, or maintain them in the guest list as they so choose.

#Microsoft #SQLServer is coming to Linux?

SQL Server is coming to Linux?

Sounds logical, given that we've already been informed parts of the Office 365 backend are Linux. See our earlier post: Microsoft and Linux: Sign of the End Times?

I'm glad to see this happening, also because it potentially makes it so much easier for one of those migrations where we read legacy calendar server databases directly and migrate them into Exchange -- you know:  

Oracle Beehive to Exchange

or Apple iCalendar to Exchange

Easy way of finding Oracle Calendar meetings without organizers post-migration to Exchange

You're migrating Oracle Calendar Server to Microsoft Exchange or Office 365 and you want to deal with the situation of Meetings without Organizers.  See our earlier post.

Given that we can run an analysis to identify these meetings before your migration you have your choice of informing your users about specific meetings (to make it as easy as possible), or handling it afterwards.  Both is probably good.

Easiest way to find them post-migration is to search on the "Meeting_MissingOrganizer" category in Outlook.

But if you are using Office 365 or on-premises via OWA instead of Outlook there's no easy list display.

You can can however find all of these visually using categories.

So set a specific color looking for the category we added on insertion, as so:

And you'll get an easy-to-locate display like this:

Again, better to get the word out to your users before the migration and get them used to the way things work in Exchange as opposed to Oracle Calendar Server, but we're all about the reality of the situation.

Oracle Meeting Scheduling Practices and Microsoft Exchange Best Practices

There are several common OCS meeting scheduling practices we need to document, so you, in turn, can educate your users about the differences between scheduling meetings in OCS and Exchange.

FIRST:  Oracle Calendar Server allows a user to organize a meeting and then remove themselves as an attendee.

THIS BEHAVIOR IS IMPOSSIBLE IN EXCHANGE! In fact if users manage to figure out how to do it’s one of the ways to really damage calendar data in Exchange.  We feel it our duty to recommend our customers adopt Microsoft Exchange best practices.  Thus, we do not recommend this behavior get replicated in Exchange.  To help you find and fix those meetings, we have integrated a pre-processing diagnostic step to both diagnose and remedy the issue. 

What it does is to re-create ICS files for the affected users and their meetings.  These files are then inserted using the standard Sumatra process.  The reports allow you to proactively notify or involve any critical calendaring users that they are going to be added to the meetings they though they removed themselves from.

How this works / what you need to do

1.   Export ALL your ICS files into a single directory.  You need to do this anyway for the migration process.

2.   If you create a separate directory that contains the oCalreader, please configure it, and point to the ENTIRE export of ICS files created in step 1

3.   Press the Organizer not Attendee button. Note: this might change in some versions of the tool.  You will have to check “Show Migration Steps”, and then you can click the Organizer button.

                                                        

4.       The tool outputs things in TWO directories

a.    The “Logfile” directory contains three files: the summary text file, along with the two CSV files for the Organizer not Attendee accounts.  These tell you which meeting organizers are affected by this situation.

b.    A subdirectory of the iCalData path gets created called “SpecialICSUsers”. The tool regenerates the ICS files that contain JUST those problem meetings -- and adds the organizer back to the meeting.  This is where the newly generated ICS files are stored

5.      It is possible there are situations where not all meeting organizers’ ATTENDEE record can be found.  Look in the SpecialICSUsers subdirectory, and see if there are any files that start with “_noattendee_”. You will have to figure out how to handle these users/meetings.  For example, if you might see a file called “_noattendee_sarah.jane.smith.ics”. You will have to either have the organizer to the meeting by hand, ask the organizer to add him/herself to the meeting in OCS and then re-export the ICS files, or choose not to migrate the meeting by deleting the files.

6.      oCalReader also checks for missing or invalid email information.  The resultant file is written to the “logfile path”, and called “AccountMissingOrInvalidEmail.txt”.  Read and act on these in advance of your cut-over into production.  For example:

CN=Clara Oswald:mailto:""

CN=Companions Conf Rm:mailto:""

CN=Companions Conf Rm:mailto:100000518943623636038552@email.invalid

CN=Medieval History Room:mailto:""

CN=Medieval History Room:mailto:182D1D7DF8E9ECA5E050C68489657375@email.invalid

CN=Martha Jones:mailto:""

CN=Martha Jones:mailto:100000250843623636038553@email.invalid

                                               

The logfile shows two users, Clara Oswald and Martha Jones, no longer have email addresses (are they terminated accounts?)  It also shows two rooms with problems.  You will have to add those rooms, “Companions Conf Rm” and “Medieval History Room”, to the resources map file to map those accounts to valid SMTP addresses.

7.       Finally to insert meetings where oCalReader has added the organizer back to the meetings, follow these steps:

a)       In the ocalreader directory include your accounts and resource mapping file(s)

b)       Edit the oCalreader Configuration

i.      Change the ICS data file directory to the “SpecialICSUsers” subfolder

ii.     Ensure there are no “limits” set

8.   Push “Read and Insert”

Note these assumptions:

• We add the organizer back to the entire series to preserve recurrence patterns and the integrity of the meeting -- even if the organizer cancels their presence on some (but not all) of the instances.  Implication: There is the potential for duplicated meetings if some occurrences do have the organizer present.)
• We set the organizer to ACCEPT the meeting (that happens by default).  Because the organizer removed themselves from the meeting we set their Free/BUSY status to FREE UNLESS the organizer has set the series to BUSY (then it becomes busy).
• For the very curious this is a full ICS generated by this process.

SECOND:  Oracle Calendar Server allows resources to be meeting organizers.

In Exchange unmanaged public resources should NOT be organizers, but we relax this in certain cases to migrate data.  Why should unmanaged public resources not be meeting organizers?  Resource accounts are DISABLED by default.  As with many rules there are exceptions.  It is perfectly acceptable to have a room direct booked or managed by a delegate (for example: “CEO’s Private Conference Room”).

If you choose to allow direct booking into conference rooms:

Exchange, by default, strips both the subject and owner in the resource calendar

If you wish to retain this information (which is common in Oracle calendar)

You will need to execute this command for those resources: 

set-calendarprocessing  : -deletesubject: $False -addorganizertosubject: $True

This has privacy repercussions!  Showing subjects and organizers reveal potentially sensitive information, such as (these fabricated examples): “Interview James Bond to replace Provost”, “Implications of the 2019 500% Tuition Hike on staff reduction plans” booked in the “President’s Conference Room.” 

You have been warned.

If you do want to determine which resources organize meetings, the “Organizer as Attendee” component generates a file, ResourcesAsMeetingOrganizers, which lists of those accounts. For example, if the Doctor as a designate to the “Tardis Control Room” proposed a meeting on behalf of the “Tardis Control Room”, called “How to use the New Dimensional Portal Controls”, the ResourcesAsMeetingOrganizers file would contain:

CN=Tardis Control Room:mailto:doctor@drwho.timetraveller.org

THIRD:  Oracle calendar server allows resources emails to be assigned to a user account.

In Exchange resources have their own SMTP accounts.  OCS allows the OCS administrator to assign a designate/delegate email to the room (so messages are sent to him/her. For example, the Tardis Break room has an email address that belongs to Amy Pond

ORGANIZER;X-ORACLE-GUID=269167A2DA4992BCE050C6848965230C;CN=Tardis Break Room:mailto:amy.pond@drwho.timetraveller.org

                                                                      

What are your choices?

1)       REMOVE the email address from the OCS rooms and re-export the data;

2)       Leave it unchanged, the designate will become the organizer of the meeting, something that they will not be happy to see on their calendars.

3)       Figure out how big a problem this is.  The “Organizer as Attendee” button generates a file, ResourcesWithUserEmails , that shows the list. For example, the file would contain:

CN=Tardis Break Room:mailto: amy.pond@drwho.timetraveller.org

How to fix: 'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine

Our full-state calendar migration executes on a 32-bit or 64-bit architecture.  We default to using the JET database engine, though this can be changed in _Config_XML:

 Provider=Microsoft.ACE.OLEDB.12.0;Data Source=

If you don't have the x64 version of MS Office 2010 installed, (or NO version of office installed) download the Microsoft Access DB Engine 2010 Redistributable (pick the x64 version!)

http://www.microsoft.com/download/en/details.aspx?id=13255  Install the x64 version via the Command Prompt with: 

AccessDatabaseEngine_X64.exe /passive

Do it this way OR you'll see the error:  The 'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine.

The error will look like this in the error window of our migration tools:

 If you look in _Config_XML you will see the default (top line) as well as the other options should you need to change them.

Provider=Microsoft.ACE.OLEDB.12.0;Data Source=

Provider=Microsoft.ACE.OLEDB.12.0;Data Source=

PROVIDER=Microsoft.Jet.OLEDB.4.0;Data Source=

On .NET Framework 4.6.1 and Exchange DANGER!

Please read On .NET Framework 4.6.1 and Exchange compatibility and then DO NOT install .NET 4.6.1.

Step away from the installer.....

Because if you allow it, to quote Microsoft "Mailboxes are quarantined and databases fail over unexpectedly in Exchange Server 2013."

Follow-on to January 1, 1970 and your iPhone

In the annals of "why there?" starting dates for calendar systems, the iPhone is not alone.

Consider UNIAPI_TIME for Oracle Calendar System (a relic of when it was Steltor and prior to that CSandT).

So if you're using OCS and do a UNICPOUTU export (aka the OLD way of doing a full-state calendar migration with us) you'll see lines in the data files that look like:

K Events:

S 9699600

D 30

T J2J

G CR Mozart

I 0

R N2

M Hendrix Jimi

W Hendrix Jimi

A TRUE 0 10

C [ .+] Hendrix Jimi

C [ .-] Garcia Jerry

O 

That "S" entry is the start time measured as the number of seconds since January 1, 1991 at 00:01:00 AM.

And of course you need to be careful of this since the great North American DST redefinition.