Calendaring Incident in Office365 March 15

Sometimes clients wonder about our paranoia religion of checking and re-checking everything in migrations.

This is the latest battle story of WHY.

Last night as we were running a migration we started noticing that invitations were not being sent for migrated meetings.  Appointments and regular email seemed to be going in fine.

This in our world is a first-rate fuster-cluck disaster.

First we naturally suspected a bug in our own code or throttling changes.  This led us to a code review, soul-searching, and much sweat-producing anxiety.  We did solve this, and it was enlightening.

Wait for it.

Office 365 ITSELF was DOWN for all practical calendar migration purposes.

We never START with blaming Microsoft. It is with great regularity their fault, but it's better practice to look at ourselves first.

So adieu this weekend's migration into Office 365, we'll have to reschedule since as of this morning this is what the situation looks like in Service Health.

Was there any kind of alert to either THEIR administrator or say OUR Office 365 administrator that this was going down?  No.  I suspect there is some way to configure these, but am perfectly willing to give someone else credit for showing me how.  Right now with the tension buzz just coming down I feel more in need of documenting the situation.

To the credit of the Office 365 team they did acknowledge that the system was throwing defecation at the ventilator.

The way this presents itself in a migration is that invitations will seemingly disappear. (Amended from earlier posting where I showed a DIFFERENT issue.)

Oracle Beehive Calendar to Exchange Migration

With three requests (two of which we consider serious) we're working on an Oracle Beehive to Microsoft Exchange migration path.

And we're going to leave all the data in your hands and simplify our usual process.
Want to get in on the ground floor?
Drop us a line.

SuHoliday cmdlet and Exchange 2007

We've gotten a few inquiries about running the SuHoliday cmdlet on Exchange 2007 instead of Exchange 2010.

Short answer: Yes you can do it.  Nothing the the was we use the EWS API is specific to Exchange 2010 to the exclusion of Exchange 2007.

However, setting Permissions in Exchange 2007 IS VERY different from Exchange 2010, so you will have to adjust for that.  Taking all of the Exchange 2007 permissions stuff out of the documentation made it substantially shorter and clearer,

If you need help look at posts in this blog involving Permissions and Exchange 2007.

Read "Freedom for Users, Not for Software"

We've been watching the migration to the cloud for a while now -- wondering when everybody was going to wise up that data centers were farming them into cruelty-free meat by-products.

I was recently at a Christmas party with some people from a Redmond-based software corporation that makes Exchange.  Their take: Office 365 made their lives and their customers lives much more convoluted.  Where they could work solutions in on-premises servers, any changes to Office 365 need to be escalated at the corporate level.  And we all know how convenient and easy that is.  So they're increasingly seeing combined Office 365 and on-premises Exchange environments, precisely the opposite of what they and the customer predicted or wanted.

SO it's is with great fervor that I suggest you read Freedom for Users, Not for Software by Benjamin Mako Hill.

He hits it right on the money with his analysis of the market confusion initially arising from "free software" which was re-cast as "open software" (goals with which it is hard to disagree! What's so funny about peace, love, and understanding?) and the way this term was used and abused in the industry.  The aspect that I suggest you pay closest attention to is the emphasis on users. Focused on the server-side of the client-server model, we at Sumatra would substitute the term "consumers" for "users" to avoid the further linguistic confusion that comes from the distinction between "users" and "administrators" in such environments.  Both the admin and the user are consumers, and the user-admin collective together face the "user" conundrum.

After years in this business, I'm pretty sure the dynamics of the industry are never going to allow the ideals of the open software movement to be fully realized in any software that is both marketable and useful.  The lure of dollars is too strong.  When software remained the exclusive domain of academics and cowboys it was possible.  These guys were happy to have a car and an apartment.

But once venture capital and the stock market took hold these ideals were not going to stand up to the motivation of owning a private jet and a McMansion.

What's this have to do with the movement to the cloud?  It's all the same dynamic based on much of the same software with the scions of the same corporations promising freedom while actually building feudal digital fiefdoms.  Do not go mindlessly with the flow when you hear that your support problems are going to go away and your life is going to be easier.  You might luck out, but really look at what your business goals are and how you're going  to deal with realistic software disaster scenarios while your business processes are directly under someone else's control.

As we often quote Ronald Reagan: "Trust, but verify."

Retiring Meeting Maker Migrations December 20, 2013

Folks, it has been a great evening out but now the bartender is calling for last orders.

As of December 20, 2013 we will no longer be doing Meeting Maker migrations.  We of course continue to do both full state and faster-simpler migrations from Oracle Calendar to Exchange, and faster-simpler migrations from Zimbra to Exchange.  Our Holiday cmdlet continues to thrive and our custom calendar engineering is unparalleled.

To those of you who have been asking us about migrating in the last few years: we'd recommend starting your test cycles sooner rather than later.

We're sympathetic to your budget cycles, so if you need to discuss timing we're open to that.  But we do need to put a stake in the ground so we can all move on.

Exchange 2010 Permissions Debugging Protocol updated

It being a new year and we having found a few new ways that Permissions could be problematic in Exchange, we've modified the Debugging Protocol.

The latest is available at this link: Exchange_2010_Permissions_Debugging_Protocol.pdf and supersedes all earlier versions.

Changes mainly affect the holiday cmdlet.

Turn on calendar publishing so people can share information

In general we don't just re-publish other links, but this one is so useful I feel the need.

How do you make your calendar available on Office 365 to users external to your organization?

Check out this article: Turn on calendar publishing so people can share information

A darned useful ability if you use it well.

#Exchange #PowerShell cmdlet for server-side holiday insertion updated

We're happy to announce we've updated the trial version of our Microsoft Exchange cmdlet for inserting holidays server-side.  This works on Exchange 2010 and Office 365.

You can read about it and get a download link by going here and filling out a form (we want to keep track of who is using it).

You can read the documentation here (and this includes how to insert class schedules using the tool as well as how to use the web-based version we've done).

We have had tens of thousands of people using this version since last year, but only a handful actually made any donations to the project.  So we're setting up a payment mechanism for the non-trial version.  When that's ready we'll let you know.

Oracle Calendar / Zimbra to Microsoft Exchange faster/simpler trial.

We've been getting lots of requests for the simplified method of going into Exchange from Oracle Calendar System and Zimbra.  So you can just download the trial here.

This version is quicker and simpler than our full method, and it will work for you IF:

• You are not changing your base SMTP addresses for users in your migration
• You do not want your recurrence patterns re-created
• You have no need of re-mapping email addresses of a subset of your users 
• Your users can deal with responding to their invitations again

If you really want the full-state, map-able, recurrence pattern recreating version you'll still need to do a trial.

Holidays into #Exchange and #Office365 via Server-Side Tools

We've just upgraded our cmdlet for inserting holidays server-side into Exchange 2010.  We'll give you some more details about that in a later post.

Mainly we're writing because we've just completed a site that lets you upload holidays into Office 365 without the cmdlet.

The full URL is: http://www.sumatraresourcewatch.com/holiday

We're treating it as a beta right now -- but we want folks to try out the online experience since it's one of the easiest ways to actually see how the full cmdlet works.  The "Buy" button will not be active until next week or so.

We've tried to make the web interface as easy as possible.  Folks who have been through a migration with us will also recognize our 1930's Soviet tractor style of user interface (not pretty but it works -- if the market responds well to this we've already got plans to update it). And since we traditionally use Jimi Hendrix as a sample account we are announcing this shortly after after his birthday.

You will need credentials for a service account that has Impersonation permissions to any of the accounts you want to modify.  This will be no surprise to any Exchange administrators.  The full manual for the cmdlet goes over how to do this.  

Then you select if you want to insert into a single calendar or upload a list of calendars to insert.

And the default calendar list goes right into Jimi's calendar.

Note that you can also remove inserted holidays automatically but leaving all other calendar data intact.

The web-based version allows you to do select a list of SMTP addresses to insert at once.  In the unlicensed version we limit this to 25 users per day, and only allow you to select US or UK holidays that we have already defined.

The full cmdlet allows you a lot more functionality: multiple time zone support, script-ability, and LDAP query insertions.

For the record, US Federal Holidays came from http://www.opm.gov/operating_status_schedules/fedhol/2013.asp  Our default file will insert JUST THIS LIST (i.e., NO “Day after Thanksgiving”) and we left Veteran’s Day a ”Free” day rather than a Busy day,

UK bank holidays came from this list https://www.gov.uk/bank-holidays 

Beta sites wanted for injecting holidays into Hosted Exchange / Office 365

Wanted: sites willing to try web application to insert holidays server-side into Hosted Exchange / Office 365.  Short learning curve (this is a web app and not the cmdlet) but we do want some feedback.  Apply to email address in above image.

Injecting Student Class Schedules into Exchange Online

At one site we've been injecting class schedules and recruiting events into individual student calendars for a few years now -- and we've just found out some others want to do it.

So here's a quick primer of what is possible.

We built a special-purpose tool for this particular client, but you can use our holiday cmdlet with the schedules in a CSV file.

Here is Zyg's daughter's Economics class in the syllabus and translated into a CSV.

Because her university probably won't give me credentials, I loaded it into my calendar (initially blank)

Then populated with her classes:

Opening one of them gives some more detail as well as seeing it tied to a useful Category.

We also have the capability to selectively REMOVE these classes, set them as Free or Busy, all the usual sorts of things you'd want to do.
Keep in mind - this is all going on server-side with no end user intervention.

The Legacy of BPOS when you migrate to O365

Had an interesting one come over the fence the other day.

A site is migrating into Office 365 but HAD been in BPOS before they started the migration process.  They were having problems with their conference rooms.

Conference rooms and resources were validating fine but no data was coming into the calendars for them.

Why should this be, we wondered?

Turns out it was an artifact of their BPOS installation.

They had the rooms set up in BPOS previously and had deleted them.  However, creating what they thought were new rooms (just with the old names) resulted in SMTP addresses like:

G9c5........a5962b@company.com

instead of 

Room_101@company.com

The alternate names were validating but email was being lost in the bowels of hosted Exchange when we tried to actually insert (remember our main recommendations: test, test, test). 

The first time when they created the rooms with the OWA user console (log in and go to Options > See all options > Manage my organization > Mailboxes). Rooms created this way using the same name values that were previously used in BPOS (say Room_101), would have  the email SMTP value assigned as Room_101@company.com and the UPN as the secondary.  BUT when you checked the same account name in the Admin console, only the UPN would be present. 

To make a migration work in this situation you have to give the rooms a NEW unique name that was never used for the SMTP address to be assigned as the primary for users to see the calendars.  In setting up your MM_Exchange_User_Map for these resources, copy the UPN name assigned as the account name and use that SMTP address.  You still have the hellacious hexadecimal as a UPN, but the method will work.


Worked like a champ.

And remember to use the UPN when setting defaults for rooms prior to migrating.  I.e., you want to NOT enforce a horizon and you want to allow conflicts.  Example:


PS C:\Windows\system32> Set-CalendarProcessing 
     -Identity  G9c5........a5962b@company.com
     -AutomateProcessing None
     -EnforceSchedulingHorizon $false
     -AllowConflicts $true



And to see what your settings are:

PS C:\Windows\system32> Get-CalendarProcessing
     -Identity   G9c5........a5962b@company.com  | fl


RunspaceId                          : 304a36b7-NNNNNNNNNNN
AutomateProcessing                  : None
AllowConflicts                      : True
BookingWindowInDays                 : 180
MaximumDurationInMinutes            : 1440
AllowRecurringMeetings              : True
EnforceSchedulingHorizon            : False
ScheduleOnlyDuringWorkHours         : False
ConflictPercentageAllowed           : 0
MaximumConflictInstances            : 0
ForwardRequestsToDelegates          : True
DeleteAttachments                   : True
DeleteComments                      : True
RemovePrivateProperty               : True
DeleteSubject                       : True
AddOrganizerToSubject               : True
DeleteNonCalendarItems              : True
TentativePendingApproval            : True
EnableResponseDetails               : True
OrganizerInfo                       : True
ResourceDelegates                   : {}
RequestOutOfPolicy                  : {}
AllRequestOutOfPolicy               : False
BookInPolicy                        : {}
AllBookInPolicy                     : True
RequestInPolicy                     : {}
AllRequestInPolicy                  : False
AddAdditionalResponse               : False
AdditionalResponse                  :
RemoveOldMeetingMessages            : True
AddNewRequestsTentatively           : True
ProcessExternalMeetingMessages      : False
RemoveForwardedMeetingNotifications : False
MailboxOwnerId                      : _4th Floor
Identity                            : _4th Floor
IsValid                             : True

As always, this is why we stress testing early and often.

Train hard -- fight easy.

Exchange 2013 First Insertion

So we're running Exchange 2013 and have successfully inserted data using our process.

Throttling looks different and we'll need to deal with it.  No surprise there.
But the early results with 2013 are very positive.
We do not expect to be needing to do migrations into 2013 for 6-12 months after final release to manufacturing (this is the historic lag we've seen with other Exchange releases), but it's good to know it's not looking like a problem.

Retiring Exchange 2007 Migrations

As of now we're retiring migrations into Exchange 2007.

There's been exactly ONE this year (which actually started in 2008 -- long story).

So unless you have a really good story or a really fat checkbook -- look to 2010 or 2013.

Exchange 2010 Permissions Debugging Protocol (revised)

Everything in the calendar migration business is a moving target: the legacy systems, the target systems, and all the permissions and APIs associated with each side.

As we never tire of saying Permissions are the biggest problem we see and the best responses to permissions issues are always fluid.  Too fluid and complex for straight blogging. To better inform our migration clients we're now extracting the Permissions section of our manual and putting it as a PDF here.

Exchange_2010_Permissions_Debugging_Protocol.pdf

We'll update the PDF as we need to -- the date in the footer being your indication that anything has been updated.

In tribute to the immortal Julia Child (whom I would occasionally see around Cambridge back in the day): bon  appétit!

Legal Discovery in Archived Meeting Maker Data

Necessity truly is the mother of invention.

I know, cliche does not become us, but it's apt in this case.

The topic is legal discovery.  A client has scads of backups of Meeting Maker server data back to the turn of the century and needs to do legal discovery on it. 

Our previous modus operandi required a full calendar insertion (meaning significant computer lab resources and expertise) and a PST export.  But after noodling on this we thought -- let's use our tool that we originally built for Zimbra insertions!  It produces ICS files which we inserted into Zimbra but are also readable in Outlook and as text!

Here's how it works:

We convert Meeting Maker export into our intermediate Access format and run zinsert (either for one user or the entire server of users if you prefer).

If you use a text editor to look at them they look like this (readable, but let's face it, geeky):

If you list them as an external file in Outlook (which does not involve IMPORTING them) - they look a lot more like calendars:

and can even be searched and displayed as a list:

Where did the HTML in my note body go?

One of our customers asked us to put up a custom message in the Sumatra Holiday tool.  This message needed a small bit of HTML formatting in EWS Managed API v1.2 code.  A "slam-dunk."  Yet this simple HTML produces horrible output:

Microsoft published a KB article that confirmed the problem:  meeting request that you send from an EWS application is in plain text format instead of HTML format when an attendee opens the request by using Outlook in online mode

The fix? Patch your servers! Here's the link: Update Rollup 3 for Exchange Server 2010 Service Pack 2 (KB2685289)

We patched our dev server this weekend and confirmed it works!

(PS: we're tagging this as an API bug, but it really isn't.....

Week of June 18, 2012 in Sumatra-land

Since as usual we have a number of sites at various stages with their purchase orders and test cycles, and Murphy's Law says they will all start to come to us at the worst possible time, a word of timely warning:

The week of June 18 Sumatra is going to be slow in responding.  So please do not expect immediate turnaround on a sudden Meeting Maker conversion or latest software for an Oracle migration.

This is due to various moving, graduation, and vacation schedules among out staff (which seem to have all hit at once).

Visual Basic Script to make calendar objects public

Well those fine, fine people at Oracle make the default PRIVATE for ICS exports in Oracle Calendar Server 10.x.

SO we have options for making our process make them NON-PRIVATE if you wish.

OR if you want to migrate it as private and then change it, this VB script will help.

---------------------

Here is a VB Script that will make items public

---------------------

'Used this MSDN as a hint: http://msdn.microsoft.com/en-us/library/aa171274(v=office.11).aspx

Dim appOutlook As New Outlook.Application

Dim OutlookItems As Outlook.Items

Public Sub ClearCalendarItemsPrivateFlag()

Set OutlookItems = appOutlook.GetNamespace("MAPI").GetDefaultFolder(olFolderCalendar).Items

For Each appointment In OutlookItems

    appointment.Sensitivity = olNormal  'or olPrivate if you want to turn it back

    appointment.Save

Next appointment

End Sub