Resource Forest Redux

We just re-wrote the sections on our migration manual dealing with Resource Forests in Exchange 2007/2010 -- here's the early version

• The "User Forest" - I started with an existing AD 2003 Domain - ad03.herring.sumatra.local (windows server 2003)
  
  • Create a user account "Blarney Stone", alias = bstone in the herring.sumatra.local domain
    
• The "Resource Forest" - a new VM: "Resource" forest domain called Sherwood: ex07res.sherwood.sumatra.local. The CAS server is "ex07res"
  
  • In AD Domains & Trusts:
    
    • Ensure DOMAIN AND FOREST levels are windows 2003
      
    • Created a TWO-way: forest trust between the Resource & User forests (Sherwood to Herring) Note: A resource forest trust is a configured ONE-way trust between the Resource & User domains. If you do this, the service account won't be able to see AD, and thus won't be allowed to access anyone's mailbox.
      

Example of the TWO-WAY forest trust between the resource (Sherwood) and the user forest (Herring) (Shown from Active Directory Domains and Trusts)

• In AD Users & Computer on the RESOURCE FOREST ("sherwood"):
  
  • Added the computer ex07res to built-in group windows authorization access group
    
  • Create a service account deleg8 in the resource forest (A new USER account).
    
• Use Exchange Management Console to:
  
  • Create LINKED mailboxes "Blarney Stone" alias = bstone (in sherwood.sumatra.local) Linked to bstone (in herring.sumatra.local)
    
  • Remember to reconfigure IIS to use SSL and have OWA default site property (in the server configuration) to use forms-based authentication
    
• In AD Users & Computer on BOTH the RESOURCE FOREST ("sherwood") AND on the USER FOREST (herring):
  
  • Right-hand click on the domain, get properties, and in the security tab Grant Deleg8 FULL ACCESS to AD. You'll have to go into advanced and set these permissions "for this object & all child objects". If you don't see the security tab, turn on Advanced Features under the View menu.
    

Example of granting FULL Control to this object & all child objects (Deleg8 on Sherwood

(Shown from Active Directory Users & Computers)

• In Exchange Management Shell, on the Resource Forest (sherwood), run this against your CAS server, "ex07res"
  
  • Add impersonation between the (resource forest) service account AND the user account:
    
    • Add-AdPermission -Identity (Get-ExchangeServer -Identity "ex07res").Identity -User sherwood\deleg8 -ExtendedRights ms-Exch-EPI-May-Impersonate, ms-Exch-EPI-Impersonation, send-as, receive-as -accessrights genericall -inheritanceType All
      
    • Add-AdPermission -Identity "Blarney Stone" -User sherwood\deleg8 -ExtendedRights ms-Exch-EPI-May-Impersonate, ms-Exch-EPI-Impersonation, send-as, receive-as -accessrights genericall -inheritanceType All
      
  • Grant Full access to the (resource forest) service account AND the user account:
    
    • Add-MailboxPermission -Identity "Blarney Stone" -User sherwood\deleg8 -ExtendedRights fullAccess -InheritanceType All
      

      
      

• In the Sumatra UI on a 32-bit machine:
  
  • Run the code as the resource service account (sherwood\deleg8)
    
    • I assume you've already granted that account local login rights, and made it a local administrator so you can read/write from the disk)
      
  • The forest: "herring.sumatra.local"; the SMTP domain: "sherwood.sumatra.local"
    
  • CAS server: ex07res (https://ex07res/ews/exchange.asmx)
    
  • Access calendar using: IMPERSONATE
    
  • Test user: bstone (SMTP address: bstone@sherwood.sumatra.local)
    

    
    

• Other Notes and Deviations from the Sumatra documentation:
  
  • Something changed between Exchange 2007 RTM and SP1/SP2. we've had to change our process.
    
  • Microsoft's David Sterling said that EWS expects there to be some sort of AD object in the resource forest to represent the cross forest account, and unfortunately, a foreign security principal is not enough. He wrote out instructions here: http://msexchangeteam.com/archive/2008/04/18/448727.aspx. BUT it doesn't work because he recommends duplicating a SID between the User and Resource forests. That generates lots of AD errors for that service account, and breaks OWA access (as that service account).
    
  • The tool to set permissions on the RESOURCE forest (Sherwood) MIGHT cause you problems because it does not explicitly set permission inheritance. So the permissions might allow you to validate against the mailbox, but NOT insert calendar data. Here was the tool: http://msexchangeteam.com/files/12/attachments/entry447730.aspx
    

• Use the Get-Mailbox -resultsize unlimited add-mailboxpermission to set permissions for all accounts, e.g., Get-Mailbox -resultsize unlimited Add-AdPermission -User sherwood\deleg8 -ExtendedRights ms-Exch-EPI-May-Impersonate, ms-Exch-EPI-Impersonation, send-as, receive-as -accessrights genericall -inheritanceType All
  

  PowerShell example of using get-mailbox (you might see warnings if you've already applied the ExtendedRights to some mailboxes.
  

  
  

• We set AD access on both the RESOURCE and the USER forests
  
• We were able to add a test item using impersonation. Delegation was not working.
  
• After the migration:
  
  • Remove the service account's full access permissions in AD
    
  • Set the trust back to a one-way trust
    
  • Remove the service account
    

• Other fun facts about resource forests:
  
  • Full Disclosure: I am not a fan of Resource Forests. Yes, they offer additional security. At the cost of 4x the complexity. I apologize to you who have implemented them successfully and are happy Exchange Admins. I'm not alone in that opinion. How a resource forest can make you cry is Vermyndax's rant.
    
  • It's easy to implement the Resource Forest in a way that causes the end user's lots of pain. For example:
    
    • Every time the user logs in to Exchange, they have to enter their resource forest credentials. That's almost as bad as my car: it automatically locking the doors once the car starts moving. Great for safety. But, every time I want to exit the car, I either have to either unlock the door before I can open it, OR pull the door handle twice – the first time UNLOCKS the door, the second time OPENS the door. Great security design. Miserable user experience. But I digress. The way around this, by the way: You have to assign the account in the USER forest these additional rights:
      
      • "Read Permissions",
        
      • "Full Mailbox Access", and
        
      • "Associated External Account"
        
    • We had problems when some DELEGATES tried to access their boss' calendars and could not. We discovered those delegate mailboxes did not reside on the same server as their boss's mailbox. The solution: move the delegates mailbox!
      
    • There were access problems for customers who have public folders (you need them if you have Outlook 2003, or if your organization uses public folders). I couldn't figure out how to solve the access problem. Thankfully Jim McBee "Mostly Exchange Web Log" AND Jesper Bernle's Exchange Server blog wrote about how to solve it. Jim McBee found and fixed issues with permissions and delegate mailboxes.

Remove "TargetAddress" in Exchange 2007

Forwarding meeting requests is a really bad practice.

See Outlook Meeting Requests: Essential Do's and Don'ts if you want the Microsoft dogma on it.

This is especially true in calendar migrations when you're creating more meeting invitations at once than you probably ever will again.

It is especially true when Exchange is forwarding automatically by defining TargetAddress (as could be done via an identity-management tool).

Such is the lead in to why you want to turn off TargetAddress before a migration.

To turn this attribute off where it is set, you simply need to follow this three step program:

1. ) EXPORT your directory data using LDIFDE

ldifde -s YOURSERVER -r "(targetaddress=*)" -l "dn" -f exportldf.ldf

This produces a file whose records look like this:

dn: CN=Adam Ant,OU=Client85,OU=Clients,DC=ex2007,DC=sumatra,DC=local
changetype: add
-

2. Edit the LDIFDE file "exportldf.ldf"
Replace "changetype: add" with "changetype: modify\ndelete: targetaddress\n\-\n\n"

i.e., the file should look like this:
dn: CN=Adam Ant,OU=Client85,OU=Clients,DC=ex2007,DC=sumatra,DC=local
changetype: modify
delete: targetaddress
-

Notes:

• If you want to restore the attribute after the migration, make a copy of the exportldf.ldf file before editing it!


• The editing works best if you have a text editor that supports regular expressions. We recommend TextPad.


• There must be a dash and a blank line between each command.

3. Re-run (import) your file
ldifde -s YOURSERVER -f exportldf.ldf –i

Why are we not doing this using either PowerShell or some of the other GUI-based management tools in Exchange 2007? Simply because we haven't yet figured out how to or if it's even possible.

Flashback: Those of you who have been through a migration into Exchange 2000/2003 will recognize the exact same worry with AltRecipient.

Cross-Forest Impersonation

David Sterling has blogged about configuring resource forests. His recent post, Cross Forest Exchange Impersonation - where the rubber meets the road (http://msexchangeteam.com/archive/2008/03/24/448500.aspx) provides examples of how one actually configures impersonation permissions. If you want an overview of cross-forest impersonation, David blogged it here.

Everything I've read from David is always well written and informative. His support for the development community is equally exceptional. Thank you, David!

Why can't I create mailboxes in Exchange 2007 via VMware

I thought it would be simple to create an additional user mailbox in our Exchange 2007 server running in VMWare virtual server. The mailbox creation process fails with this message: "An Exchange 2007 server on which an address list service is active cannot be found."



Why do simple tasks that fail drive me nuts, and take forever to resolve?

What broke? The Microsoft Exchange System Attendant failed to start! Once restarted, I created the mailbox with ease. This was the same thing I saw when creating mailboxes for an Exchange Resource Forest Trust. So for whatever reason, the SA continues to randomly fail to start. A three minute task stretched to fifteen. Reminds me of the joke... I always give 100% effort at work: 10% on Mon, 25% on Tue, 35% on Wed, 25% on Thu, 5% on Fri.

Trust Me (Part Four) - I see a Resource Forest thru the trees

What's the old expression -- you can't see the forest for the trees? Well, on my fourth attempt, I can.

To recap: What was the problem? I wanted to insert calendar data into Exchanged 2007 that was configured using a Exchange Resource Forest topology. The user mailboxes were linked to a accounts in a user forest. Those user mailbox accounts were disabled. You can not use Exchange Web Services to impersonate a disabled account.

What worked? Here is what I did:

• Upgraded Exchange 2007 to SP1,
• Created a service account in the resource forest
• Gave that service account DELEGATE permissions to ALL of the disabled accounts.
• Granted permission for the service account to see the user forest's AD
• Inserted calendar data!

Finding the disabled accounts, and assigning delegate rights to the service account turned out to be easy, thanks to Jian Li’s MSExchange Team Blog “How to access multiple resource mailboxes in Exchange Web Services (EWS)”. He described how it works, and (even better) created a script that has worked well for us (get the Microsoft Script.)

Oh, for the last few days the Exchange System Attendent decided to automatically start, and remain running. No idea why.

-R

Trust Me (Part Three)

Ran into two issues when configuring the resource forest and the user forest: service accounts and Address List System Service failing.

Service Account: I created two service accounts, one on each machine. The service accounts had the same name. When I went to link the resource account to the account in the user forest, accessing the linked domain controller failed.

Lesson learned - The link failed because I didn't qualify the service account with the user forest (e.g. userforest\mySvcAccount). So Exchange tried to link to the user forest domain controller with the resource service account and failed. Had I created service accounts with two different names, this problem would not have tripped me up.

Address List:

After I fixed the service account issue, the last step of in the "linked mailbox" wizard failed with an invalid address. After much head-scratching, I discovered the system attendent had failed with an MSExchangeSA event ID 1005: "Unexpected error The Local Security Authority cannot be contacted ID no: 80090304 Microsoft Exchange System Attendant occurred. "

Dave Goldman blogged about this: Creating a new mailbox in Exchange 2007 with the new-mailbox cmdlet fails with Address List Service not Available, but only addressed one aspect. My problem was choice (4) - SA stopped. I haven't figured out why it's failing, but restarting the SA allowed me to complete the mailbox link.

OK--everything is set. Next time I'll report on the results of impersonation.

Trust Me (Part Two)

This is the second installment on my experience trying to setup a resource forest trust.

Last night I re-installed VMware's Server product on an Windows Server 2003 SP2 box that also had Exchange 2007 SP1 installed. I rebooted the box and left. I lost today's popularity contest to "Smiling Bob" after our early-morning users received "Service Unavailable" when they tried to visit our intranet, OWA, and anything that used Exchange Web Services.

What happened? The Application Event Logs show W3SVC-WP Event IDs 2268 and 2214, and System Events IDs 1002, 1039. It looks like VMware Server and Exchange 2007 are both trying to control IIS (VMware wins....)

To restore the server, I uninstalled VMware Server and reset IIS to run in native mode using this AdsUtil command line:

cscript %systemdrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32BitAppOnWin64 0

BTW, if you want to re-enable 32 bit apps (WOW64 Mode), run this command:
cscript %systemdrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32BitAppOnWin64 1

It was suggested that VMware's WORKSTATION product would be a better choice. I just installed the evaluation version. 'Sure n b'gora' ...web services and the intranet are still available. (Thanks, Chrisopher Q., for the tip!)

Now it's back to trying to build a one-way resource forest-trust.
-R

Trust me

We are testing Exchange 2007 in our 64-bit Windows Server 2003 Exchange Resource Forest Topology using two virtual servers (with VMWare -- Microsoft Virtual Server does not work in a 64-bit environment). We created two VMs -- the "user forest" and the "Exchange forest". We set up a one-way trust between the two forests, and tried to create a "Linked" user mailbox.

After a head-banging week, we can NOT get this to work. No matter what we do, when we try to link to the mailbox we get a Event ID 2130 (MsExchangeADAccess....Exchange Active Directory Provider could not find an available domain controller). We can ping the domain, the nsLookup is fine, dcdiag /v, netdiag /v -- everything comes back without errors.

Is anybody else having this problem? How did you solve it?

One of Sumatra's clients suggested this might work if we created five VMs - one for the DNS/DHCP, a second for the user DC, a third for the resource DC, a fourth for Exchange, and a fifth for the client. We will test his theory later this week (or this weekend).

Here is an image of the topology linked from the Microsoft Exchange Resource Forest Topology article:


-R