Monitoring ADFS through AAD Connect Health Agent

The AAD Connect comes with a Health Agent which monitors the AAD Connect and logs in to Azure AD.

The events can be viewed from portal.office.com and Choose Azure Active Directory. Under Manage -> Choose Azure AD Connect.

On the right pane, under Health and Analytics -> Click Azure Active Directory Connect Health

Now that we know how AAD Connect is being monitored by AAD Connect Health Agent, the same agent can also be used to monitor ADFS server and ADFS-Proxy server as well.

To do so,

1. Download the AAD Connect Health Agent from https://www.microsoft.com/en-us/download/details.aspx?id=48261
2. Run the setup, make sure you are installing ADFS agent.
3. It will prompt you to “Configure now”.  Click on it.
4. You will see the following

Make sure you have enabled auditing on ADFS server to capture those events on Azure portal.

To do so,

1. Open windows powershell on ADFS server,
2. Run the following command

auditpol.exe /set /subcategory:”Application Generated” /failure:enable /success:enable

3. Open AD FS management console, Click on “Edit Federation Service Properties”, Under Event tab, check all the events.

Once you’ve done that, you can see events will start showing up on Azure Active Directory Connect Health

Of course you can monitor lot of events.  To know more,

https://docs.microsoft.com/en-us/azure/active-directory/connect-health/active-directory-aadconnect-health-adfs

VJ

 

.Net framework update causes AAD Connect to crash or CPU goes up to 100%

A recent .Net frame work update is causing the AAD Connect server to crash or the CPU utilization goes up to 100%

Go to Task manager and check the list of processes,

If MIcrosoft.Online.Reporting.MonitoringAgent.Startup is consuming high CPU or

If you have the following update,

KB4338420	Windows Server 2008
KB4338606	Windows Server 2008 R2
KB4054542	Windows Server 2012
KB4054566	Windows Server 2012 R2
KB4054590 KB4338814 KB4338419 KB4338605 KB4345418	General

Then update your Azure AD Connect Health Agent as soon as possible to avoid or stop a major issue in your infrastructure.

Use following link to know how to update them,

https://docs.microsoft.com/en-gb/azure/active-directory/connect-health/active-directory-aadconnect-health-agent-install#download-and-install-the-azure-ad-connect-health-agent

Reference: https://support.microsoft.com/en-gb/help/4346822/high-cpu-issue-in-azure-active-directory-connect-health-for-sync

 

Cannot add new connectors – AADConnect

AAD Connect is very flexible when the organization expands but at the sametime we should make sure the AAD Connect is up-to-date to coupe with the changes.

One of the incident that I’ve encounter may help others.

Issue : Cannot add new connectors to AAD Connect

I wanted to add a new forest to sync up to the Azure AD. All these users are going to be in the same office 365 Tenant.

Symptoms:

When i add the connector (Microsoft way – Using AAD Connect wizard), I get the following error

E_MMS_SCHEMA_CLASS_NOT_FOUND

When i check the logs,

Troubleshooting:

• Ports
  • Checked the ports to see if the domain is reachable
  • Made sure the domain name is resolved to the right ip’s
• The schema made be feel that, the service account doesn’t have rights on the forest that is being added
  • Checked the permisisons –  it is all good
  • Changed the permission to Domain admin

• Adding new connector
  • Added a new connector manually (click the new connector from Synchronization service console)
  • Pointed to the forest
  • When i did the right-click and “Search Connector space”, i can see all the objects from the domain (it wont sync anyway as the sync rule wont get populated, if you use “Create” connector)

At this point, i understood that it is not a problem with the forest that i’m trying to add.

I ran the wizard without adding the connector, i got the same error.  So this proves that there is an issue with the existing connector.

Resolution:

I ran refresh schema on all the existing connector.  I found one of the connector had a schema changes which wasn’t picked up by the AAD Connect (One of the forest administrator installed Exchange server in their infrastructure)

After refreshing the schema, i ran the wizard, it went like a charm.

 

 

Password hash sync is not working for sub-domains – AAD Connect

Issue: The password sync for sub-domains are not working

Data Collected:

1. The password hash sync for the root domain and selective sub-domains are working without any problem
2. The user and other objects from the selected OU of the all the root domain and the sub-domain works without any issues
3. There is no sync errors for the object which doesn’t sync the password
4. When a password has been reset for the object from the sub-domain, there is no event id 656 or 657 logged on the AAD Connect server
5. Properties of the connectors shows that sub-domain Directory partition has been checked.

Troubleshooting:

Before I proceed, I have done everything mentioned in the article below,

https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnectsync-troubleshoot-password-hash-synchronization

1. Checked whether password hash is enabled – It is
2. When I run the following command,

Invoke-ADSyncDiagnostics –PasswordSync

From the following screenshot it shows that the sub-domains directory partition not considered as domain.

3. Ran the above command against the object in the sub-domain for which the password sync is not working

Invoke-ADSyncDiagnostics -PasswordSync -ADConnectorName <Name-of-AD-Connector> -DistinguishedName <DistinguishedName-of-AD-object>

If you look closely, it says that it is available in metaverse database but an error for the objects of the sub-domain

“There is no password has synchronization rule for AD Connector space object”

4. But that’s not right as you can see from below screenshot, There is a sync rule for “In from AD – User AccountEnabled” is true

 I didn’t bother to get deep in to the sync rule as the installation not customized.  I was sure that the domain partition is not recognized

5. The domain partitions are selected in the connector properties.

To check this,

• Right click on the connector
• Choose properties
• From the popup window, click on “Configure Directory Partitions”

I now came to conclusion that the domain partition is not recognised but from the GUI it shows it is selected.

After some googling, I found 3 interesting cmdlets

• Enable-ADSyncConnectorPartition
• Enable-ADSyncConnectorPartitionHierarchy
• Update-ADSyncConnectorPartition

There is no explanation of these cmdlets but I did manage to run it but with no success.

Resolution:

So finally I’ve gone back to basics of powershell.

Get-ADSyncConnector

This gives me list of connectors.  I need the first connector (where the sub-domain is)

$c = Get-ADSyncConnecor

I’m interested in the first connector and its partition, I’m assign that into the variable

$adConn= $c[0]

$AdConn.Partitions

This will list down the list of partitions under that connector.  There are about 5 partition, out of that last 2 partition’s object is having problems

If you closely look in the attribute called “IsDomain” is set to “False”, but the same is “True” for the rest of the domain partition (Its not in the screenshot though)

This exactly the same reason when we ran the password sync troubleshooter, it said that the sub-domain in questions is not a domain

To change this value, run the following command, for 2 sub-domains

$adConn.Partitions[5].IsDomain=$true

$adConn.Partitions[6].IsDomain=$true

After the change it will look like below

We are not done yet.  This should take care of the password sync but

Set-ADSyncAADPasswordSyncConfiguration -SourceConnector “domain.net” –TargetConnector “domain.onmicrosoft.com – AAD” -Enable $false

Set-ADSyncAADPasswordSyncConfiguration -SourceConnector “domain.net” –TargetConnector “domain.onmicrosoft.com – AAD” -Enable $True

Soon after this, the eventlog shows lot of 656 events indicating that password sync of the objects from sub-domains are syncing.

 VJ

 

Unable to login to Office 365 Tenant

I’ve encountered an interesting issue that may be well useful to troubleshoot and how to fix it

Issue : Users are unable to login to http://portal.office.com.  They get the following error

Analysis :

The error message itself is not very useful.  So, i had to collect few information to know what is going on

1. AAD Connect for that tenant is still running.  There is a recent delta sync success in the logs
2. Everyone including the Global administrator cannot login
3. I can login with the cloud-only account
4. Checked with other tenants, and there is no news from MS on the message center (this rules out if this is a Microsoft issue)
5. Password change seems to be replicating.  There are some logs for the users who recently changed their password
6. The AAD connect version is 1.1.5

The information collected was not so very useful at all except one thing

Full/Delta Synchronization is working without any issue

Troubleshooting:

Ran Fiddler from the machine  where i tried logging to office 365 and found an interesting information.

If you closely look at it, the authentication is being redirected.

So, this generally happens if there is an ADFS server.

There is another instance that this could happen which is “Pass-Through Authentication” Feature enabled on AAD Connect.

To know more about it,

User sign-in with Azure Active Directory Pass-through Authentication

https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnect-pass-through-authentication

Long story short about the process,

1. The user tries login to https://portal.office.com or trying to access any office application
2. Once the credentials are provided by the user, Azure AD, on receiving the request to sign in, places the username and password (encrypted by using the public key of the Authentication Agents) in a queue.
3. An on-premises Authentication Agent retrieves the username and encrypted password from the queue.
4. The agent decrypts the password by using its private key.
5. The agent validates the username and password against Active Directory by using standard Windows APIs, which is a similar mechanism to what Active Directory Federation Services (AD FS) uses. The username can be either the on-premises default username, usually userPrincipalName, or another attribute configured in Azure AD Connect (known as Alternate ID).
6. The Agent cannot send evaluation request back to Azure AD and failed to respond if the authentication is (success, failure, password expired, or user locked out)

Solution:

The pass-through authenticate is failing for some reason.  This proves the fact that “Full/Delta Sync” is working because the service account is cloud only account.

Disabled the pass-through authentication on AADConnect using AAD Connect wizard and revert it to password-hash synchronization