Mailboxes could not be synchronized

Error description:

a dtrace log shows us the following information:

616    15:18:09.423     [5952]    (AgentClientBroker)    <8096>    EV:M    CAgentExchSynch::SynchronizeMailboxEntryV9 - Start to synchronize mailbox /o=heerwegh/ou=users/cn=Recipients/cn=olivier
616    15:18:09.423     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::GetMapiSessionFromPoolEx(AdditionalFlags = 0)
617    15:18:09.423     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CreateMapiSession - Entering Routine
618    15:18:09.767     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CreateMapiSession - Calling CreateProfileAndSessionEx
619    15:18:09.767     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - ProfileName=VaultMbxAgent-EVSERVER1-8096-1320416289-0-767-0
620    15:18:09.767     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - CreateProfile
621    15:18:09.782     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - CreateMsgService
622    15:18:09.829     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - GetMsgServiceTable
623    15:18:09.829     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - QueryAllRows
624    15:18:09.829     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CPAS() - ConfigureMsgService
625    15:18:09.829     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Trying ConfigureMsgService using PR_PROFILE_UNRESOLVED_SERVER=[EVSERVER1], Attempt=[1]
626    15:18:09.845     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - ConfigureMsgService failed with error [0x8007000E]
627    15:18:10.345     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Using Provided Global Catalog Server=[DC0002.heerwegh.ch]
628    15:18:10.345     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Trying ConfigureMsgService using PR_PROFILE_UNRESOLVED_SERVER=[DC0002.heerwegh.ch], Attempt=[2]
629    15:18:10.360     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - ConfigureMsgService failed with error [0x8007000E]
630    15:18:10.860     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Trying ConfigureMsgService using PR_PROFILE_UNRESOLVED_SERVER=[EVSERVER1], Attempt=[3]
631    15:18:10.876     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - ConfigureMsgService failed with error [0x8007000E]
632    15:18:11.376     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Trying ConfigureMsgService using PR_PROFILE_UNRESOLVED_SERVER=[DC0002.heerwegh.ch], Attempt=[4]
633    15:18:11.392     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - ConfigureMsgService failed with error [0x8007000E]
634    15:18:11.782     [5952]    (AgentClientBroker)    <6036>    EV:M    CSecurityWrapper::CSecurityWrapper
635    15:18:11.782     [5952]    (AgentClientBroker)    <6036>    EV:M    CSecurityWrapper::HasServerClientGotPermission, Admin Operation: 53
636    15:18:11.782     [5952]    (AgentClientBroker)    <6036>    EV:M    CSecurityWrapper::IsServerClientTheVSA
637    15:18:11.782     [5952]    (AgentClientBroker)    <6036>    EV:M    CSecurityWrapper::IsServerClientTheVSA - hr=0 InClientCOMCall:T VSA:T
638    15:18:11.782     [5952]    (AgentClientBroker)    <6036>    EV:M    CSecurityWrapper::HasServerClientGotPermission: VSA: Allowed
639    15:18:11.892     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Trying ConfigureMsgService using PR_PROFILE_UNRESOLVED_SERVER=[EVSERVER1], Attempt=[5]
640    15:18:11.907     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - ConfigureMsgService failed with error [0x8007000E]
641    15:18:11.907     [5952]    (AgentClientBroker)    <8096>    EV:M    ConfigureMsgServiceAttempt - Exiting [0x8007000E]
642    15:18:11.907     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CreateProfileAndSession - Call to ConfigureMsgService Failed [0x8007000E]
643    15:18:11.923     [5952]    (AgentClientBroker)    <8096>    EV~E    Event ID: 3411 An error has been reported by ConfigureMsgService.  It may be due to a problem contacting a Global Catalogue Server. |Exchange Server: EVSERVER1 |Mailbox: SMTP:ev_sys_2010@heerwegh.ch |Error: 0x8007000e   |
644    15:18:11.923     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CreateProfileAndSession(VaultMbxAgent-EVSERVER1-8096-1320416289-0-767-0) - Profile failed (0x8007000e)
645    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMailboxHelper::CreateProfileAndSession - Delete Profile (0x0)
646    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CreateMapiSession - Release Mutex(2)
647    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::ClearProfileCache()
648    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CloseMapiSession - Entry
649    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CloseMapiSession - [1]
650    15:18:11.938     [5952]    (AgentClientBroker)    <8096>    EV:M    CMAPISession::CloseMapiSession - [2]

Symptoms:

Are the following conditions met ?

• The MAPI error code that is mentioned in the Dtrace log is : [0x8007000E]
  • This error code means : "not enough memory". If you got another MAPI error code, resolve it using this link: List of Extende MAPI numeric result codes
• In the eventvwr there are a number of errors. The first error you get when you run a synchronizing / archiving task is event 3411.
  • The error message states: "An error has been reported by ConfigureMsgService.  It may be due to a problem contacting a Global Catalogue Server."
  • This is a misleading error message as it is in fact not related to global catalogue server problems.
  • event 3188 occurs directly after event 3411. This error is saying "not enough storage"

Solution:

• Restart the "Enterprise Vault Task Controller Service" daily using a batch file. This service contains all the archiving tasks in Enterprise Vault and messaging agents.

  • net stop "Enterprise Vault Task Controller Service"
    net start "Enterprise Vault Task Controller Service"
    

  • If you want to restart the EV tasks and agents in a more granular way, then use the "evservice" command in the EV directory.
• Configure the HKLM\Software\KVS\Enterprise Vault\Agents\ProfileExpire registry key:
  • Default the value is set to 3 days. change it from 3 to 1. This way, the MAPI profiles in outlook will expire faster and corrupted profiles will get removed so they don't keep on using valuable memory / resources.
• Increase the server performance
  • The errors occur on very busy ev servers. probably, you have a 32 bit server running already with 4 GB installed memory?! If the SQL server is installed locally, the SQL server role should be moved to another node or you should use boot ini switches to increase the addressable memory limit from 3.xx to 3.xx.