Sep 26 2022 03:59 AM - edited Sep 26 2022 04:00 AM
You can now validate the compatibility of your SCIM provisioning endpoint and Azure AD code base using our Azure AD SCIM Validator. This tool can be used by ISVs who want to build SCIM compatible servers either for gallery app or generic app and developers building their line of business SCIM apps. https://learn.microsoft.com/azure/active-directory/app-provisioning/scim-validator-tutorial
Nov 29 2022 05:05 AM
Where's the best place to ask questions about the results I'm getting from the validator? My POST /Users request is failing validation but I'm not getting a reason for the failure
Nov 29 2022 07:15 AM
Turns out the status code should be 201, but the validator doesn't indicate that's the problem
Dec 19 2022 07:58 AM - edited Dec 21 2022 07:32 AM
Can the SCIM Validator be made open source or at least have a location where issues can be filed against it? We've encountered a couple problems with the SCIM Validator and would like some way of reporting issues or discussing misunderstandings with the developers.
The first is that it does not read the schema correctly. For instance, our endpoint describes the 'email' 'primary' attribute as boolean, but SCIM Validator sets it to string, apparently ignoring our schema.
Another is a failure during group testing where it re-creates a user that already exists. It seems not to expect a 409 reply.
It is very hard to tell *what* the tester expects without a diagnostic error or insight into its source code.
Dec 20 2022 08:41 AM
@Vincent_Drake_Cerberus I'm so glad I'm not alone, and want you to know that you're not alone either! I've spent the past few days trying to figure out what I'm doing wrong. I finally reached the same conclusion that the validator doesn't expect a 409 to be returned after looking through my server logs.
The validator makes a patch request to change a username from 'x' to 'y' as part of a completely different test, and then as part of the failing test, the first thing it tries to do is submit a post request for a user with username 'y'... I can get it to pass by removing the duplicate user throws an error code (and just returning the user instead), but then the Create a duplicate User test obviously fails.
Very strange that the otherwise randomly generated test data is kept between tests.
Dec 21 2022 07:23 AM
Jan 10 2023 06:01 AM - edited Jan 10 2023 06:22 AM
This feedback is noted @Doogal @Vincent_Drake_Cerberus and @JameyAbdelnoorJameyAbdelnoor . We are working to address these, and I will share the individual updates.
Jan 30 2023 12:04 PM
@owinoakelo , same issue as @Vincent_Drake_Cerberus with phone primary attribute being sent as random phone number string instead of boolean value.
Feb 06 2023 05:09 PM
Feb 06 2023 11:00 PM - edited Feb 06 2023 11:04 PM
Looks like there was recently a big update on the Validator. The "Discover Schema" now sets the correct type (string/boolean). Thank you!
However, now we're seeing a new complication. It appears the Validator is trying to add new entries using replace with a value selection filter (valuePath). This was discussed in 708183: SCIM PATCH of Complex Multi-Valued Attribute Includes Filter and Sub Attribute in Path in the context of patching with add. And that discussion makes sense as the spec is a little vague.
However, with replace, RFC 7644 is quite specific in 3.5.2.3:
o If the target location is a multi-valued attribute for which a value selection filter ("valuePath") has been supplied and no record match was made, the service provider SHALL indicate failure by returning HTTP status code 400 and a "scimType" error code of "noTarget".
If the valuePath does not match anything, then you need to return 400 noTarget. Microsoft Azure AD SCIM Validator does not appear to expect that reply and expects us to add new entries on replace.
I disagree that:
o If the target location path specifies an attribute that does not exist, the service provider SHALL treat the operation as an "add".
would change this interpretation as that does not consider the filter; plus the way these are written they get more and more specific as you go down the list.
A specific example:
The current value of emails:
"emails": [
{
"type": "work",
"value": "some_primary_email",
"primary": true
}
and the replace operation:
"Operations": [{
"op": "replace",
"path": "emails[type eq \"home\"].value",
"value": "some_email"
}]
NOTE: values do not show email addresses as those get removed (privacy checker) - so they are replaced with "some_primary_email" and "some_email"
This should fail as there is no email of type = home to replace.
Feb 07 2023 01:41 AM
@Vincent_Drake_Cerberus We released a fix that addresses the read schema issues. The new release also addressed the duplicate creation the triggers the 409 error.
To the question on having the making the validator open source, we do not have that in the current plan. We are working on a channel to enable you raise issues.
Feb 07 2023 03:27 AM
Feb 07 2023 07:59 AM
@owinoakelo Thank you for your fast reply.
To clarify, are you saying that using PATCH replace to add multi-valued entries is:
a.) what Azure's SCIM implementation currently does as of Feb 2023 (we have not seen this in practice)
or
b.) an issue with the Azure AD SCIM Validator only
If option "a", is this affected by the feature flag _aadOptscim062020_? And if so, does the Validator accept/use those feature flags? (I wasn't able to see a difference with/without the flag after the last update.)
Feb 15 2023 06:57 AM
@PatM_Cerberus Yes, the azure provisioning service currently uses replace for patch operations and expects the endpoint to add if needed. That's why the scim validator makes the same request and it supports the capability of the feature flag.
Feb 16 2023 12:11 PM
@owinoakelo We have been working to allow Azure's specific replace semantics.
We've noticed when using Schema Discovery, the system now correctly captures that primary is a Boolean.
phoneNumbers[type eq "work"].primary boolean
However, when it replaces phoneNumbers, we're seeing:
{
"Operations": [
{
"op": "replace",
"path": "phoneNumbers[type eq \"work\"].value",
"value": "767.814.5465"
},
{
"op": "replace",
"path": "phoneNumbers[type eq \"work\"].primary",
"value": "(974)693-6276 x6968"
}
],
"schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ]
}
Notice that the boolean primary value is a text string. We specifically have handlers for Azure sending "True" or "False", but this value is not acceptable and we're failing it.
I believe this is a problem with the Azure AD SCIM Validator, but curious if Azure AD would try to send such a bad value.
Email values now seem to work properly - we're getting true/false booleans for primary in those.
Feb 22 2023 06:19 AM
@PatM_Cerberus , you are right. This is an issue with the SCIM validator. The Azure code base will respond with a Boolean as expected. We have added this in our backlog of fixes and will address it. Thanks for highlighting this.
Mar 16 2023 06:55 AM
I suspect I found a bug in SCIM Validator. What should I do?
Please refer to the image below.
Mar 16 2023 08:24 PM
Just running through the validator, and noticed that for CREATE operations, when the status code returns 200 it fails the test, however the test result doesn't inform us that it's looking for a 201 status (we had to try a lot of things before we figured it out)
Mar 20 2023 05:40 AM
@norfas Thanks for highlighting this. This has to do with the communication of the error. Question, when you did the patch operation, did you commit the updates that we sent over to your db? This error is mostly likely coz you did not commit the updates and so what you returned to us in the shown response is not the true state of your db.
Mar 20 2023 09:19 AM