Migrating iPlanet Calendar to Exchange via SuPump

Over the last five years of doing calendar migrations we've heard of pretty much every legacy calendar system out there. One that comes up now and again is iPlanet, once bundled with Netscape (remember them?).

iPlanet has a tool: csexport.

The file extension of the OUTPUT file (.ics or .xml) determines the format it will export to: ither iCalendar or XML

So since suPump.exe takes an XML file as input for incorporation into Exchange, this gives a quick way of server-side migration.

You use it as follows:

csexport -c jsmithcal calendar jsmith.xml

then

suPump.exe /k /AT:UserID /u:jsmith /DT:A /file:"jsmith.xml"

Will add appointment data for a UserID where the XML data is in file “calendar.xml”. This command does not clear the calendar calendar before inserting data.

Sumatra Pump Contact Schema

SuPump.exe uses the following fields for inserting contacts. Size limits are up to available limits in Outlook.


ContactID: alpha numeric identifier (can be left blank)
ContactNum: Contact Number
OwnerUserID: UserID or mailboxAlias
FirstName:
LastName:
Title:
Dept:
Room:
Company:
Address:
City:
State:
Zip: XXXXX-XXXX
Country:
Email:
HomePhone: XXXXXXXXXX
WorkPhone: XXXXXXXXXX x XXXXX
Fax: XXXXXXXXXX
MobilePhone: XXXXXXXXXX
Pager: XXXXXXXXXX
OtherPhone1: XXXXXXXXXX
OtherPhone2: XXXXXXXXXX
OtherPhone3: XXXXXXXXXX
Custom1
Custom2
Custom3
Custom4
NoteInfo :additional information

If an attendee to an occurrence of a recurring meeting is removed then the series is updated, the removed attendee is re-invited.

Chris Quinn from Milwaukee writes that:

If an attendee to an occurrence of a recurring series is removed.

Then the series is updated.

The removed attendee is re-invited and the meeting shows back up on their calendar.

The following steps can be followed to reproduce the error.

1. Create a recurring meeting with “Organizer 1” and “Attendee 1” for at least 2 days.
2. Open any occurrence of the meeting and remove “Attendee 1” and send the update
3. Open the series and make any modification(e.g., Add a note to the details) send the update.

You should notice that “Attendee 1” still has the meeting on their calendar, however when you open the occurrence of the meeting that had “Attendee 1” removed, you will notice that “Attendee 1” is not in the meeting.

Chris further adds:

This is not a problem in CDO 1.21 .... This is a problem in Outlook. When a recurring series is updated it does not send the correct updates for the occurrences.

This is similar to a problem Microsoft has acknowledged with resources of which we had written earlier:

We've had reports of meeting locations being overwritten to exceptions to
recurring meetings. As Microsoft KB Article 916108 shows, it is an acknowledged bug in Exchange.

Sample Syntax for Exchange Insertion with the Sumatra Pump

To read and insert a file contacts.xml into “Sumatra.com”. Tags the items with the default keyword
supump.exe /dn:Sumatra.com /file:contacts.xml /k

Insert one appointment for USERID=37238L. Remove data prior to insertion. Read source data from http (oracle server).
supump.exe /UNDO /k /AT:userID /u:37238L /DT:A /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_calendar?userid="

Appends one appointment for ExchangeAlias T_L to existing data (missing /undo)/ Read data from http.
supump.exe /k /AT:Exchange /u:T_L /DT:A /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_calendar?userid="

Remove user’s 37238L’s Appointment data that is tagged with the default keyword
supump.exe /dn:Sumatra.com /undo /k /u:37238L

Insert appointments for USERIDs in a text file. Remove appointment data prior to insertion. Read data from http.
suPump.exe /UNDO /k /AT:UserID /in:"F:\Sumatra\Clients\UserIds.txt" /DT:A /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_calendar?userid="

Insert appointments for ALL exchange users with USERIDs in a text file. Read data from http. Remove appointment data prior to insertion
suPump.exe /UNDO /k /AT:Exchange /ALL /DT:A /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_calendar?userid="

Add appointment data for one UserID where XML data is in file “calendar.xml”. Clear calendar before inserting data.
suPump.exe /undo /k /AT:UserID /u:37238L /DT:A /file:"F:\Sumatra\Clients\calendar.xml"

Insert contact data for ALL User IDs. Clear contacts before insertion. Read data from http.
supump.exe /UNDO /k /AT:Exchange /ALL /DT:C /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_contact?userid="

Insert Task data for User IDs in a file. Clear task data before insertion. Read data from http.
supump.exe /UNDO /k /AT:UserID /in:"F:\Sumatra\Clients\UserIds.txt" /DT:T /http:"http://oracle-dev.site.edu/dart/harpo/" /PKG:"EXCHANGESYNC_PKG.run_task?userid="

Insert all data into the domain name “lab.sumatra.com” for aliases defined in the file IT_Test.txt. Obtain the data from the web, and tag it with keyword testrun4
Suxml /dn:lab.sumatra.com /k:testrun /u:37238L /k:testrun4
/HTTP:"http://oracle-dev.site.edu/dart/harpo”
/pkg:"EXCHANGESYNC_PKG.run_calendar”

Reports of problems with solution to KB 932511

Chris Quinn, one of our brethren calendar and scheduling wonks, has reported an additional problem with the solution to KB article 932511: Exception meeting requests are deleted from the calendar in Outlook 2003 when recipients use a CDO 1.21 application to accept the master meeting request


This of course comes up because of the DST issue in North America, in his case he had an exception to a meeting that had been scheduled to another resource. The fix on 932511 and the Rebaser put the exception back on the ORIGINAL resource.

We'll keep you updated on if there is a fix to the fix to the fix (we think the recursion only goes that far, but after three we just lose interest and move on to other projects).

Chris also reports that:

Because we forced all the updates we also found out about another
problem. We found that OWA views can differ significantly from the Outlook
views. So we found another article that was quite interesting.
I thought you'd get a kick out of it. Now we can all POOF our calendars…
http://blogs.msdn.com/stephen_griffin/archive/2007/02/21/poof-your-calender-really.aspx

NB: To all of your looking to migrate Proxies and Designates from legacy calendar servers to Delegates in Exchange -- we need to use CDO 1.21 to accomplish this and the acknowledged memory leaks (see KnowledgeBase Article 891509 A memory leak condition occurs when you create a CDO program or a MAPI32 program to log on or log off many mailboxes in Exchange Server 2003 SP1) are one reason we treat this very carefully. See also KBs 901014 and 913643.

Syntax for SuPump.exe

The Sumatra Pump (which on occasion we pronounce "SumPump") uses the following syntax (which will be familiar to all of you using the Sumatra Utilities):

suPump /dn:domain /sn:servername /k:kwd /all /u:usr /in:infile /at:acct_type /dt:data_type
/out:outfile /undo /v:file /http:url /pkg:p
where:

/dn:domainname =Run on this "domain name"

/sn:serverName =The server name

User
/at:acct_type=Account type (acct_type=Specific or Exchange) default is Exchange alias.
/all=Retrieve ALL Aliases with SpecificIDs (employeeID)
/in:infile=user INput file, one Exchange Alias or SpecificID per record (default infile.txt)
/u:user =process this Alias or SpecificID User (supersedes /in:)
SpecificID in this case means anything you want to define, an account number or employee ID for example.

/dt:data_type=Data type = Activities, Tasks, Contacts (default is Activities)

/k:kwdstem = Assigns a category during an insert. E.g., /k=holiday2006; Also limits mtgs/appts/contacts/todos tagged with Keyword stem e.g. /k:ca finds ca,car,cat (/k default:mmconv102659080256Z).

/out:outfile=Output file (default outfile.txt)

/undo=remove items from calendar (tagged w/ keyword). Items to be removed is a function of the /dt

Data Location
/file:filestring=passes an XML file (path included) to be processed
/HTTP:URL=Obtain a file to be processed from HTTP address using this URL root. Note, the URL must be enclosed in quotes
/PKG:Pkgname=defines the package name to extract the data. The package will be appended to the URL root to retrieve the data. The package name must be enclosed in quotes.

Sumatra Pump: The Third Way to get Calendar Data into specific accounts

In Exchange there are two ways to get data into a user calendar:

1. Invite the user to a meeting which they accept
2. The user or their delegate types an entry themselves

This was too restrictive for a major graduate business school in the Northeast. There was no way for users to "subscribe" to calendar events relevant to their needs, such as course schedules and assignments, on-campus recruiting, and then have those events appear on their calendars.

They asked Sumatra to develop an Exchange server-based tool we refer to as the Sumatra Pump, suPump.exe.

A bit of background

This school built a web-based calendaring solution that offers its students, faculty, and staff a single source to access all academic calendars and schedules including the academic calendar, class schedules, club events, and one-on-one meetings such as the interview calendar. The web-based solution also provided users with a list of contacts and tasks. Users can personalize the solution by setting preferences to select the calendar data they wish to receive. End users wanted the calendar data in Outlook, so the school built an Outlook add-in that allowed users to merge selected calendar data into their Outlook calendar.

Fine and good, but this was manual -- the user had to initiate the command sequence. The school’s IT department wanted a solution that could be scheduled each night to push the requested data into Outlook.

Sumatra automated the calendar data merge process by implementing a server-based “Push” product on Exchange. The goal of this Push product is to deliver calendar, task, and contact updates to Outlook end users on both a scheduled basis and an on-demand basis.
This solution had the following results:

• Consolidated scheduling data delivery at the Exchange server level;
• Reduced system maintenance and management costs;
• Minimized the negative productivity impact of requiring users to initiate the calendar merge sequence, or by re-keying calendar data into Outlook;
• Extended support to all Exchange clients on-site not served by the client-side VB macro then in use (including, for example, OWA and Entourage users);
  

The suPump takes as input an XML file. For Activities the file has the following structure:

ActivityID: alpha numeric identifier (can be left blank)
UserID: SchoolID or mailboxAlias
Name: Activity name
Location:Activity location
Banner:True/False - Is this an all day event
Private:True/False - Should this be marked as private
Label:What color label (0-11) corresponds to Outlook labels
Duration:Duration in minutes
ActivityTime:Activity start date and time
LastTime:Activity end date and time
TimeZone:timezone (see below)
FreeBusy:"Free", "Busy", "Tent","OOF"
RecurType: Not implemented for this client
Ongoing:Not implemented for this client
DaysMask:Not implemented for this client
WeeksMask:Not implemented for this client
DaysFromEndOfMonth:Not implemented for this client
Period:Not implemented for this client
Exceptions: Not implemented for this client
Reminder: number of minutes
NoteInfo: additional information

Contacts and Tasks have similar formats.

We've been thinking of using this for simple calendar / contact / task migrations for formats we do not currently handle (iPlanet calendar migration is one of the cases occasionally requested).

This is also something we've looked at as part of a co-existence strategy for alternate calendar servers and Exchange.

Using the Sumatra Utilities to archive appointments

In our continuing effort to make sure you don't think of the Utilities as something just to use to fix the looming Exchange DST crisis, we have a new feature to blog about:

archiving

Aside: you multinational companies shouldn't think it's almost over! I've gotten at least two inquiries from Europeans who are wondering about using the Utilities when the European Union reviews DST for its member nations. And "just move to Exchange 2007" is not the clear answer for everybody.

So why use the Utilities to archive? You could use automatic archiving to a PST file -- which will reduce the size of your local data and speed things. BUT if you want easy access to this data and still want to speed your calendar display, you can use the utilities to move all appointments before a certain date to a specified archive folder.

This gives you the best of both worlds: Better speed AND easy access (if you need it).

So to move all of user “riuliano”'s one-time (non-recurring) appointments before 8/2/2006 to a folder called “My Old Cal Items” use this command:

Su /u:riuliano /dt:8/2/2006 /mc /fn:"my old cal items"

If you do not have your aliases as SMTP addresses, use this syntax:

Su /fn:”my old cal items” /mc /dt /smtp /dn:lab.sumatra.com /u:russ_iuliano@lab.sumatra.com

to move appointments for user russ_iuliano@lab.sumatra.com (identified by SMTP address, /smtp) one-time (non-recurring) appointments before yesterday to a folder called “my old cal items”.