Tuesday, September 25, 2007

Converting your OCS exports to migration format

So you've gotten your calendar data out of Oracle Calendar Server and now you wonder "What ELSE do I have to do with it?"

Well, you have to convert it into a (new) relational database so that we can reassemble any attributes not exported from OCS.

Like what you ask? Guest lists, guest responses, recurrence patterns, all the sorts of things you want.

Do you need to go through this? Well, if you want to do this all server-side and keep all the information that makes schedulers valuable, yes.

The tool that takes your exports runs in Windows and looks like this: How do I use this?

While the design reminds you of an early farm tractor (which we are very proud of) - it's designed with the same single-minded focus: get the job done as quickly as possible and the heck with how it looks (you're tossing it out after this is done anyway).

The idea here is to take in the files you got from UNICPOUTU, and match them with users and resources from USERS.TXT and RESOURCES.TXT.

Specifically this will:

  • Match export data to users and resources (the users and resources will be mapped to your new calendar system)
  • Add guests to your meeting
  • Group one-time instances into “recurring meetings” (makes the insertion run faster)
  • Remove old items you do not intend to insert (makes the insertion run faster)
  • Reformat OCS-specific items to calendar-neutral terms (e.g. map Daily Notes and Daily Events to All Day Events)

Note that if you're planning on multiple test insertions you only need to read the users and resources into the database once. It is possible to parallelize conversions, e.g., convert A-L on one machine, M-Z on another and join them together to get the job done in half the time (but please talk with us before you do that).

This also includes a blank template database (MS Access format), which we recommend you rename a copy of to something descriptive, like "December 1 migration test." Please preserve the database version in the file name; it really helps when debugging.

Let's go into some details on the specifics.

Time Zone: Select the Time Zone of your OCS node. This is really crucial otherwise everything will be off on your insertion by some multiple of hours. If you have users outside of the default time zone you can modify the USERS.TXT file for them (see earlier post).

Drop Items Prior to this Date: You can selectively export items from certain date ranges -- but in case you have a data set with everything and want to pare it down for a test insertion, you have that option here.

Threshold for recurrence patterns: When we get the events from OCS they are all individual events rather than a series. We reconstruct the series, but (as a little thinking will show you) with exceptions to recurrences this can get a little dicey. Our software analyzes the sequence and comes up with its best guess of the recurrence pattern, taking account the threshold you set.

A couple of quick things to note:

  • Historical events (i.e., any events before the date you convert them) all go in as individual events (it's faster this way) with meeting guest list in the Agenda field (except for BCC guest whom only the meeting proposer sees)
  • There needs to be at least five recurring events before we attempt to re-create the sequence. (Quick math -- at an 80% threshold, one exception to a five event pattern puts you at the threshold)
  • If we can't match a pattern all events will go in as individual instances. So worst case scenario is that a recurring meeting or event comes in as a series of individual instances. (But as opposed to an ics RDATEs list these will display properly in Outlook clients).
  • If you migrate only a subset of users, meetings to and from users outside that subset will not show up. Think about this one: if the meeting owner or guest is not in the system you're migrating to -- how can they be on a meeting in your new system? This is the main reason we're always advocating "Big Bang" migrations for meeting-centric corporate cultures.

How does this work?

In two phases: the first phase reads the OCS exports and turns them into an intermediate file. The second phase reads the intermediate file into the database template.

Why not just put everything into the database as we're going along? It's faster this way.

Everything in a calendar server migration is a trade-off between time and convenience.

It's certainly easier for the average admin to work with an Access file. But the overhead of constant transactions at the volume we need is huge. Saving them for the end makes the system much faster.

No comments: