Using iTunes to sync contacts on your legacy system and your migration to Exchange

oooohhhh boy.

You think using Apple iTunes on your legacy system (in this case Oracle Beehive) to sync contacts will result in no problems (because Apple has such high quality (yes, this is sarcasm) software in the Windows world.....).  But when you migrate into Exchange some contacts are duplicated.

But you migrate and you get some duplicate contacts.  Not from ALL users, just the ones using iTunes.

Yep.  That happened in a real world site this past weekend.

Use this procdure post-migration to de-dupe:

Delete duplicate contacts

If you imported contacts into Outlook by using the same names or e-mail addresses that already exist in your Contacts folder, and you selected the Allow duplicates to be created option in the Import and Export Wizard, you might have unwanted duplicates of several or all of the contacts that you imported.

Removing the unwanted duplicate contacts is a manual process, but the following is the easiest way to do it.

NOTE:  These instructions are for Outlook 2007. Instructions are also available forOutlook 2013 or Outlook 2010.

In Contacts, select the contacts folder that has duplicate contacts.

In the Navigation Pane, under Current View, click Phone List. This is the best view to scan your contacts list and see the duplicate contacts. Now you can sort the list by modified date and group the duplicates together.

On the View menu, point to Current View, and then click Customize Current View.

Click Fields, select Modified in the Available fields list, and then click Add.

Click Move Up until Modified is at the top of the Show these fields in this order list.

Click OK twice.

In the list of contacts, hold down CTRL while you select each duplicate contact.

When you have selected all the duplicate contacts, press DELETE.

This is from:

https://support.office.com/en-us/article/Resolve-or-delete-duplicate-contacts-0fa9c0a0-499a-48ab-b818-4e6463ccb9fa

Oracle calendar to Zimbra via ICS does NOT solve all your problems....

In response to our post: Oracle + Zimbra?  we got at least one person writing in saying "But we can just import the ICS files."

Yes, you can use ICS export/import and it SORT OF works.

You will not have your recurrence patterns and every recurring meeting and appointment will come in as a series of single, disconnected instances.

That is for starters.

Then there are the issues of mapping users throughout the OCS namespace so they map to the new IDs in your Zimbra namespace.

Then there's the issues of conference room / resource behavior that OCS allows but Zimbra does not and how you should deal with that.

Oracle + Zimbra?

My latest newsletter from Zimbra had the following:

As the referenced Press Release Oracle Gives Partners a Fast Path to Cloud explains: "Zimbra plus the Oracle Cloud means users can start small and grow, or start big and get bigger"

Forget the fact that Zimbra's had its own cloud that's been promising the same thing.

The Zimbra back-end of mySQL (now owned by Oracle) makes this start to look more interesting than it might otherwise seem.

Consider this:  Microsoft has the very successful Exchange and Google has its office suite.

What Oracle has is the obsolete Oracle Calendar and the moribund Oracle Beehive.  

Given that Oracle traditionally cannot stand not having anything in the enterprise that Microsoft and Google already have. are we looking at the first tentative steps towards an acquisition?

OCS / Beehive into Zimbra is doable on your own if you want, but you'll need to be really careful of resources.  If you want to do it the right way and you've at least a few thousand users drop us a line.  We've got a long history taking both Oracle and Zimbra into Exchange (I mean, check out the rest of this blog).

After 20 Years Sega Saturn DRM Cracked.... and what it has to do with calendaring.....

Bravo to Dr Abrasive for cracking the Sega Saturn DRM method.

What on earth does this have to do with calendaring?

Well, since we broke Meeting Maker, Oracle Calendar Server, Zimbra, Oracle Beehive, and a few other encoding schemes, we're not allowed to talk about, we're sort of connoisseurs of reverse engineering and call out kudos where they are warranted.

What's kind of amazing is that none of the engineers came forward in the last 20 freaking years with anything that would help this.  Of course, for Meeting Maker the group of hacks finally working on it only started to contact us after we'd done everything significant to read their data and insert it into Exchange.

And don't even get us started about the Oracle people!!!!

Microsoft SQL going after Oracle? Don't forget the calendars in Oracle Calendar Server or Beehive!

Microsoft waves the 'free licenses' flag to try to grab Oracle database users?

Sure.

I'm not certain that the arrogance of Microsoft is preferable to the arrogance of Oracle (what's the difference between Larry Ellison and god?  God doesn't think he's Larry Ellison.).  But free is a hard argument.

Just remember we can migrate your enterprise calendars (Key word:  Enterprise-- if you have 100 Oracle calendars, just use client-side methods.  If you have 500+ users and mission-critical meetings, check us out).

What if I forgot my #Oracle BEE_DATA Credentials in a Beehive to #Office365 Migration?

Our Oracle Beehive to Microsoft Exchange migration reads the Oracle database directly, which makes things WAY easier for a migration.

Using the BEE_DATA credentials gives us access to all the data email migration products do not take.

Recently though we had someone who inherited their Beehive installation and did not have their BEE_DATA password.

Yuck!  Normally we do not like to advise things like just changing the password on a running system, since you have no idea what else it affects.  As Buckaroo Banzai advised: "No, no, no, don't tug on that. You never know what it might be attached to." 

This script will create a user called “sumatraTestBeeData” with all the privileges we need.

--------------------------------------------------------------------------------------------------------------------------

--Using SQL*Plus, 

connect sys as sysdba/PASSWORD;

select tablespace_name, status, contents from user_tablespaces where tablespace_name like 'bee_%';

drop user sumatraTestBeeData;

--create user command:

CREATE USER sumatraTestBeeData IDENTIFIED BY PASSWORD  DEFAULT TABLESPACE BEE_DATA TEMPORARY TABLESPACE temp QUOTA UNLIMITED ON users  QUOTA UNLIMITED ON BEE_DATA;

grant sysdba to sumatraTestBeeData;

grant create session to sumatraTestBeeData; 

grant ALL privilege to sumatraTestBeeData;

Grant UNLIMITED TABLESPACE to sumatraTestBeeData;

--sumatraTestBeeData needs access to these three tablespaces:

--BEE_DATA, BEE_LOBS, BEE_INDEX

alter user sumatraTestBeeData quota UNLIMITED on BEE_DATA;

alter user sumatraTestBeeData quota UNLIMITED on BEE_LOBS;

alter user sumatraTestBeeData quota UNLIMITED on BEE_INDEX;

--------------------------------------------------------------------------------------------------------------------------

Then use these Beehive credentials in bCalReader:

Full-state Calendar Migrations from #Oracle, #Zimbra, #MSFTExchange, etc into @Office365

There must be a forest around here somewhere, I just cannot see it through all these trees.

These are Sumatra's full-state calendar migrations into Microsoft Exchange that 

• re-create meetings
• keep recurrence patterns 
• preserve guest responses. 
• book conference rooms and resource
• make it like your users have been using Outlook and Exchange all along. 

• Everyone else who claims to do calendar migrations skips over all this functionality. To be fair, it is hard to execute correctly (and most of them have their hands full convincing you to do email when imapsync does it so well and so much less expensively), and it's not too important if you have a few hundred users. But it is the whole ball of wax if you're any real-sized corporation.

Exchange to Office 365

with Incremental Sync

Oracle Calendar Server to Exchange / Office 365 (via ICS files, which we still use to create live meetings):

Oracle Beehive to Exchange:

MDaemon to Exchange (we don't do full-state on this)

Meeting Maker we've been doing but we didn't do a video on it.  And at this point -- why bother?

Zimbra to Office 365:

Metrics for an #Oracle #Beehive Migration to #Office365

Our latest Oracle Beehive to Microsoft Exchange migration tool brings back the "Use Report" capability that allows you to generate an HTML summary of how much data your Beehive server contains.

Clicking the highlighted button....

Will generate a report called beehivemetrics.htm that looks like this:

 

Yes, we had some fun with how we referred to the top volume users.

This is great information for determining how to do a segmented migration across multiple CPU instances (not that we had to enable exactly this over the last weekend or anything....).

Disappearing Contact/Calendar item body fixed in Exchange 2013 CU6

We blogged about clients reporting problems with notes after inserting contacts and calendar items from Beehive and MDaemon migrations April, '14.)


KB 2975003 confirmed this was a problem! The KB says: "...compose or edit a Calendar item by using Outlook Web App...and then save the item. When you open the item in Microsoft Outlook 2013 or Outlook 2010 in online mode, the body of the item disappears."  We scrambled and updated our code to set MAPI codes to fix the problem in our code.  But they reported their end users were still having the same problem on NEW items.



Now there is a fix: install Cumulative Update 6 for Exchange Server 2013: 2961810

BTW, for  customers with legacy tools, CU6 fixes the problem MAPI/CDO clients cannot connect to Exchange Server 2013.

#Oracle Beehive to #MSExchange through a proxy -- alternate strategy

We never shy away from saying that migrations are hard.

We also never shy away from saying "Nope -- you're in over your head."

And we thought both of these were going to come to bite us as we looked at migrating Oracle Beehive from behind a proxy server.  

A set of test data we would normally expect to migrate in a few minutes was taking hours.

And there was no way to remove the proxy.

Running the calculations told us the full set of users would take four days on a single instance.  We could not put someone through that and maintain anything like guilt-free sleep patterns.  We put on our thinking caps and came up with the idea of exporting the data into a format we could use (Oracle Beehive IS a SQL database after all), taking that outside the firewall, importing it into an Oracle database, and then inserting the data (the final step herewith shown):

Problem solved.

If it's applicable to you we'll take you through it in your trial phase.

Oracle Beehive to Microsoft Exchange Calendar Migration: Beehive-Buddy

Oracle Beehive seems to be the legacy system folks are replacing at record pace these days.

So we've been burnishing some of the features.

Latest one: Beehive Buddy Lists.  

Any contacts tagged as Beehive-Buddy will not migrate.  This reduces the number of duplicate contacts.

Hosted Oracle Beehive to Office 365 Field-Proven

We had an informal bet, Zyg taking the side that Hosted Beehive to Office 365 was not going to work the first time out and Russ figuring that it was.

Russ won. 

Data goes in like a champ in the current version of the Beehive migration code.

Since we're having to go through a proxy server performance is much lower than we see in on-premises migrations -- but that is what multiple instances are for.  Stay tuned for more news.

Oracle Beehive Unexpire and Unlock Accounts

Just to record all of this stuff in the event you need it.

If your passwords in Beehive / Oracle have expired, then Oracle Unexpire and Unlock Accounts is really, really useful.

So too is Oracle Database Security Checklist.

Hard Limit of 25,000 on Beehive Calendar Objects

While stress testing our migration methods -- we did a major push for performance optimization aka SPEED -- before a migration this past weekend and of course that meant a few bugs crept in, we got this message from Oracle Beehive.

So if you did not know it before, there's a hard limit of 25,000 meetings in Oracle Beehive Extensions for Outlook.  Hard as it is to believe we do know of users who have exceeded this in other legacy systems.

External Contacts in an Oracle Beehive to Microsoft Exchange Calendar Migration

External email addresses in meeting invitations.

We do not make them live in migrations.  The reason is simple: they confuse the heck out of the attendees and it's not worth it.  While you can communicate to your internal folks that a migration is going on and we are taking care of re-creating the meetings, we cannot reach out of your organization and re-accept or decline for your external guests. 

But we can let YOU know about them.

So as of all forthcoming version of our process we're keeping the record of external guests in the meeting agenda, exactly as you see here post-migration.

When Impersonate Permissions Fail DURING a Migration

Interesting case came up this weekend.  

An 850 user Beehive to Exchange Migration validated fine (notable for two things: First European Beehive migration and first multiple instance Beehive migration) and then while in migration started throwing impersonate errors for what became a total of ten users.

Now ten out of 850 is about 1% -- which to us is still too great an error rate for us to tolerate.  

But it seemed really wasteful to undo the entire migration for ten users.

Here's what we did.

We modified the insertion to allow for a single user.  Not via validation of that user which is a good test procedure but it would be insufficient here.  You need the validation file to be as entire as you can get it -- otherwise there will be inaccuracies.

First fix your permissions issues with those users.

THE ABSOLUTELY SAFEST WAY

Here are the steps we recommend:

•     Create a new sub directory for each of the accounts.  Remember to copy configuration and the two account validation and mapping files
•       Launch the bcalreader UI, and set the lower and upper limit for the first account’s SMTP address root, e.g.:  “jimi.hendrix” to “jimi.hendrix”
•     If there are multiple accounts that have this root, follow the limit values with an “@” sign.  For example, if you have two accounts “jimi.hendrix”, “jimi.hendrix2”, and you only want to process “jimi.hendrix”, set the limits: “jimi.hendrix@” TO “jimi.hendrix@”           Otherwise the tool will migrate “jimi.hendrix” and “jimi.hendrix2”, and you will end up with duplicates in the “jimi.hendrix2”’s mailbox.
•     Validate, then press the “process all” button.
•     Repeat for other users

NOTES

• This inserts all meetings/contacts/tasks owned by the user, and process all invitations received from other users.
• However, it will NOT process invitations sent from this account to other accounts.  Those invitations will remain in his invitee’s in-boxes.  We suggest you leave it that way.
• Communicate to your end users that the migration may have left some meeting invitations in their inbox.  Suggest they process those invites the way they did in Beehive (e.g., if they accepted the invite in Beehive, accept the one in Outlook;  If they declined the invite in Beehive, decline it in Outlook.

THE ABSOLUTELY BEST WAY

The best way in this case means getting the outcome you could have had if the impersonate had not failed.  But it is a little more intricate and has the possibility you press the wrong buttons.

Do this in the cold light of morning.  It is way too easy to hit the wrong button when you are sleep deprived and stressed. 

• Create a sub-directory that you will use to process the accounts that had impersonation problems.
• Copy bCalReader v2.0.11 or greater and the validation file.
• Change the limit to the account.  Put an “@” after the SMTP address, e.g.: “Jimi.Hendrix@”
• Press “Process All”.
• Repeat for the remaining accounts.

Once you have added data from the impersonation-problem accounts, return to the “main instances” of bCalReader.

For each of these instances:

1. Check the “show individual steps”, and re-process the invitations as follows
2. Press the “Respond to Invitations” button on each instance.  When they are all done,
3. Press the “Apply response Exceptions” button on each instance.  When they are all done,
4. Press the “Update Tracking Info” button on each instance.  When they are all done,
5. Press the “Final Cleanup” button on each instance.  
6. When they are all done, you are finished! 

Multiple Insertion Instances in Oracle Beehive Calendar to Exchange Migration

Now that we have Beehive to Exchange migration running reliably and in test in both the USA and Europe, we're working on improving the throughput with our old friend: Multiple Instances aka Segmentation.

Since re-creating calendar state in Exchange requires sending, responding, and creating exceptions for meeting invitations, this technique does require some special handling, which we're detailing here.

First, the screen morphs a little:

As we've done in the past, this is an alphabetic list, so 

1. Make sure when you do this you cover the entire alphabet.  Once a site left out the letter "k" -- yet another great example of why we built "UNDO" capability into the process.

2. Sometimes resources or rooms start with numbers, so make your first installation "SPACE" instead of "A".  If you try to use A this comes up: 

3. Remember what we said earlier about sending invitations and responding to them?  Suppose that Adam Ant invited Lex Luthor to a meeting, but Luthor is in a different segmentation.  It's possible one instance could be trying to respond for Luthor before he gets the invitation and suddenly everybody gets confused.  No worries.  After the invitations have been sent for each instance the code pauses and tells you to wait for all the other instances to catch up.

How to use this:

• Go to each computer you want to use 
• Install bCalReader and its associated .DLLs
• Validate and map all your users on one machine and then COPY the account_validation.txt and alt_account_maps.txt to the other instances.
• Proceed.

As always, TEST TEST TEST before you go live.

Performance Optimization in a Beehive to Exchange Calendar Migration

We've been fine-tuning the performance in the Beehive Calendar to Exchange calendar migration.

A little bit of background is due here.  We use the rate of 850 objects per minute as a rough gauge of Insertion time into Exchange in our Meeting Maker or OCS migrations.  But that is ONLY for insertion into Exchange from our own data structures and does not include the time for any processing of Legacy calendar data (the Extraction phase).

For Beehive, we've combined the EXTRACTION and INSERTION into one application where we read directly from Oracle's relational database.

And needless to say performance varies hugely with the Beehive topology and servers.

So a few general results:

• Your migration performance improves  20-30% if you SHUT DOWN Beehive (technically our results were 18% to 33%, in a variety of configurations, but rounding makes them easier to wield).
• Going into on-premises Exchange yields more reliable performance than insertion into Office 365.
• Extraction involves a certain amount of overhead that is difficult to remove and highly dependent on Beehive system and configuration.  Think of it this way: it always takes a certain amount of time to pull up a user.
• Given that, look at your summary report for you top users and use that data in conjunction with your test experience to gauge your insertion times.  If you need help analyzing this contact us.
• Calendars with more objects take longer to insert so only take data as far back as you need.  If we see serious requirements for lots of past data we'll modify the code to allow you to insert current data as quickly as possible (so you are functional) and then go back and insert your historical archive.  This has been a regular feature of our other migrations for a while now, but Beehive migrations are relatively new.

More #Oracle Beehive Calendar Metrics

Our latest version of the Beehive to Exchange / Office 365 migration code includes more table of metrics of Beehive calendars.  I show some of the additional stats we generate here using only two users in our test server so it's easier to take in.

#Oracle Beehive Calendar Metrics

We just cannot leave well enough alone so we started analyzing Oracle Beehive data (live, of course) to produce summary reports such as you see below in HTML.  We've been doing it for years for Meeting Maker and OCS but Beehive is relatively new in our demand-space.

If you want to try this in your environment contact us info AT sumatra DOTCOM and please tell us how you will give us helpful feedback.