Authentication fails with MSFT Provider extension, V2.0 endpoints and Graph

[decomplexity]

BACKGROUND
I am running PHP 5.6 and Steven Maguire’s Microsoft Provider extension to thephpleague’s oauth2-client in order to use the MSFT Identity Platform V2.0 authorization and token endpoints and the MSFT Graph V1.0 API with PHPMailer.
The V2 end-points and Graph are needed to support SMTP AUTH with Oauth2 (as announced in May 2020), and my MSFT tenant has SMTP AUTH enabled (MSFT is disabling it by default for new tenants).

I have made the obvious changes (below) needed to Steven’s code to support v2.0 endpoints and the Graph API v1.0.

PROBLEM DESCRIPTION
Running my get_oauth_token manually successfully gives a refresh token that is pasted into my PHPMailer invocation code.
But subsequently invoking PHPMailer results in a 535 5.7.3 authentication failure:
(A 535 5.7.3 fail code is not in RFC 4954’s code list but seems a common enough ‘invalid credentials’ error.)
I have posted variants of this problem on thephpleague/oauth2-client and stevenmaguire/oauth2-client repositories and also on the thenetworg/oauth2-azure (which has its own provider code)

DEBUG OUPUT
SMTP ERROR: AUTH command failed: 535 5.7.3 Authentication unsuccessful [AM3PR05CA0135.eurprd05.prod.outlook.com]

However: running get_oauth_token manually is recorded in AAD Sign-ins but subsequently invoking PHPMailer is not. It appears that whatever calls get_oauth_token (whether directly or by callback from an endpoint) is not doing so and hence not obtaining authentication.

I am baffled and clearly doing something daft. Any suggestions are most welcome, especially from anyone who has successfully implemented this using v2.0 endpoints and the Graph API.

CODE DETAIL

My compose.json ‘requires’ just:

 "phpmailer/phpmailer": "dev-master"     …… which should pick up PHPMailer 6.1
 "stevenmaguire/oauth2-microsoft": "dev-master"

My changes to the Steven Maguire’s Provider code are in vendor/stevenmaguire/oauth2-microsoft/src/Provider/Microsoft.php:

    public $defaultScopes = ['Mail.Send SMTP.Send offline_access openid profile email User.Read']
    protected $urlAuthorize = 'https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize'
    protected $urlAccessToken = 'https://login.microsoftonline.com/organizations/oauth2/v2.0/token'
    protected $urlResourceOwnerDetails = 'https://graph.microsoft.com/v1.0/me'

My PHPMailer invocation is:

session_start();
require 'vendor/autoload.php';
require_once('vendor/phpmailer/phpmailer/src/OAuth.php');
require_once('vendor/phpmailer/phpmailer/src/PHPMailer.php');
require_once('vendor/phpmailer/phpmailer/src/SMTP.php');
require_once('vendor/phpmailer/phpmailer/src/Exception.php');
require_once('vendor/stevenmaguire/oauth2-microsoft/src/Provider/Microsoft.php');

use phpmailer\phpmailer\OAuth;
use phpmailer\phpmailer\PHPMailer;
use phpmailer\phpmailer\SMTP;
use phpmailer\phpmailer\Exception;
use Stevenmaguire\OAuth2\Client\Provider\Microsoft; 

$mail = new PHPMailer;
$mail->isSMTP();                                      
$mail->Host = 'smtp.office365.com';
$mail->SMTPAuth = true;                  // Enable SMTP authentication
$mail->AuthType = 'XOAUTH2';             // Select OAUTH2 
$mail->SMTPDebug = SMTP::DEBUG_LOWLEVEL;
$mail->SMTPSecure = 'tls';               // Enable TLS encryption
$mail->Port = 587;
//$mail->Username = [my email address];    //only for Basic Authentication AuthType
//$mail->Password = [my email password];  //only for Basic Authentication AuthType

$username = ‘[my email address]’;    
$clientId = '[my client ID from AAD]';
$clientSecret = '[my client secret from AAD]';
$redirectURI = '[URI of my get_oauth_token.php module, as registered with AAD';
$refreshToken = '[1049 character token, extracted from get_oauth_token’s response] ';
$mail -> refresh_token = $refreshToken;

$provider = new Microsoft([
    'clientId'          => $clientId,
    'clientSecret'      => $clientSecret,
    'redirectUri'       => $redirectURI
]);

$provider->urlAPI = ‘https://graph.microsoft.com/v1.0/’;
$provider->scope = ‘openid  SMTP.Send Mail.Send offline_access profile email User.Read’;

$mail->setOAuth(
        new OAuth(
            [
                'provider' => $provider,
                'clientId' => $clientId,
                'clientSecret' => $clientSecret,
               ‘refreshToken' => $refreshToken, 
                'userName' =>$username
            ]
        )
   );
// The remaining code is purely PHPMailer and works fine with Basic Authentication

My GET_AUTH_TOKEN.PHP is:
… just a Microsoft-specific and slightly pruned version of Steven’s own. The ‘Select Provider’ is thus strictly unnecessary.

<?php
namespace PHPMailer\PHPMailer;
use Stevenmaguire\OAuth2\Client\Provider\Microsoft;

if (!isset($_GET['code']) && !isset($_GET['provider'])) {
?>
<html>
<body>Select Provider:<br/>
<a href='?provider=Microsoft'>Microsoft/Outlook/Hotmail/Live/Office365</a><br/>
</body>
</html>
<?php

exit;
}
require 'vendor/autoload.php';

session_start();

$providerName = '';

if (array_key_exists('provider', $_GET)) {
    $providerName = $_GET['provider'];
    $_SESSION['provider'] = $providerName;
} elseif (array_key_exists('provider', $_SESSION)) {
    $providerName = $_SESSION['provider'];
}

$clientId = [my client ID from AAD];
$clientSecret = [my client secret from AAD]';
$redirectUri = [URI of this module, as registered with AAD ';

$params = [
    'clientId' => $clientId,
    'clientSecret' => $clientSecret,
    'redirectUri' => $redirectUri,
    'accessType' => 'offline'
];

$options = [];
$provider = null;

switch ($providerName) {
        case 'Microsoft':
        $provider = new Microsoft($params);
        break;
}

$provider->scope = ‘openid  SMTP.Send Mail.Send offline_access profile email User.Read’;

if (!isset($_GET['code'])) {
    // If we don't have an authorization code then get one
    $authUrl = $provider->getAuthorizationUrl($options);
    $_SESSION['oauth2state'] = $provider->getState();
    header('Location: ' . $authUrl);
    exit;

// Check given state against previously stored one to mitigate CSRF attack
} elseif (empty($_GET['state']) || ($_GET['state'] !== $_SESSION['oauth2state'])) {
    unset($_SESSION['oauth2state']);
    unset($_SESSION['provider']);
    exit('Invalid state');
} else {
    unset($_SESSION['provider']);
    // Try to get an access token (using the authorization code grant)
    $token = $provider->getAccessToken(
        'authorization_code',
        [
            'code' => $_GET['code']
        ]
    );
    
    echo 'Refresh Token: ', $token->getRefreshToken();
}

[Synchro]

A few minor things first: Why use PHP 5.6, especially if you're doing new development? It's years out of date and no longer supported. Since you're using composer (you're loading its autoloader), you don't need to load PHPMailer's classes manually as well. I'd also recommend against using dev-master; stick with release versions.

I always have a hard time with OAuth at the best of times, but one thing that strikes me as odd (even as I look at PHPMailer's own examples) is why the access token isn't kept and used for auth, and the refresh token only used in case the access token has expired.

[decomplexity]

Thanks Synchro.
I have rerun using PHP 7.4 and with Composer ‘requiring’ PHPMailer 6.1.7 and stevenmaguire Provider extensions 2.2.0.
No change.

And to your comment about priming the system with a refresh token rather than an access token (either being taken from a manual run of get_oauth_token), I can only assume that this reflects the possibility that the access token will indeed have life-expired, and that using a refresh token (which has a longer – perhaps indefinite – shelf life) allows the invocation of PHPMailer to immediately use the refresh token to request an access token. But this is an assumption: I have failed to find any explanatory documentation.

[HaiderWain]

Did you manage to find a solution to this? I am having the same error. Authentication fails. I have successfully done all this with GMAIL and PHPMAILER XOAUTH2, but O365 fails authentication.

[decomplexity]

Yes - try using an O365 scope of:
offline_access https://outlook.office.com/SMTP.Send
and a URL API of:
https://outlook.office.com
AAD won't yet let you assign equivalent permissions as outlook isn't in the list of APIs (resource servers) but MSFT appear to have assigned at least an SMTP.Send permission behind the scenes.
See https://github.com/decomplexity/SendOauth2/blob/main/MSFT%20OAuth2%20quirks.md for details.

[HaiderWain]

Thanks for the fast reply, WOW!. I looked into the "MSFT OAuth2 quirks.md" note by you. Thanks for the explanation. Do you mind if I contact you by email for a few questions regarding this integration?

[decomplexity]

Fine - I'm at [email protected]

[ewalscargocare]

I have used Decomplexity's code and amendments to achieve authentication via OAuth. I do get a refresh token, but still can't authenticate:

2022-09-29 12:01:14 SMTP INBOUND: "535 5.7.3 Authentication unsuccessful [AM0PR02CA0082.eurprd02.prod.outlook.com]"
2022-09-29 12:01:14 SERVER -> CLIENT: 535 5.7.3 Authentication unsuccessful [AM0PR02CA0082.eurprd02.prod.outlook.com]
2022-09-29 12:01:14 SMTP ERROR: AUTH command failed: 535 5.7.3 Authentication unsuccessful [AM0PR02CA0082.eurprd02.prod.outlook.com]
SMTP Error: Could not authenticate.

Can you explain the solution? I did read MSFT OAuth2 quirks.md, but don't understand the solution.

[decomplexity]

Suggest using a scope of:
offline_access https://outlook.office.com/SMTP.Send
which will authenticate to the Outlook REST API V2
There is a known problem with SMTP.Send and Graph V1.0 that I am currently trying to elucidate with MSFT AAD Team.
(A scope of Mail.Send or SMTP.Send with no URI prefix will force a token AUD field (resource target) of Graph)

[ewalscargocare]

Thank you for your answer - So if I understand correctly I need to change:

ORIGINAL
public $defaultScopes = ['Mail.Send SMTP.Send offline_access openid profile email User.Read']

NEW
public $defaultScopes = ['offline_access https://outlook.office.com/SMTP.Send']

ORIGINAL
$provider->urlAPI = ‘https://graph.microsoft.com/v1.0/’;
$provider->scope = ‘openid SMTP.Send Mail.Send offline_access profile email User.Read’;

NEW
$provider->urlAPI = ‘https://outlook.office.com/SMTP.Send)';
$provider->scope = ‘SMTP.Send’;

Correct?

[decomplexity]

Almost. Suggest:
$provider->urlAPI = ‘https://outlook.office.com';
although I think that for simple SMTP sending, $provider->urlAPI = ‘https://graph.microsoft.com/v1.0'; will also work (it may not get used).

[ewalscargocare]

Unfortunately, that still doesn't work. Can you look at my code?

Microsoft.txt
index2.txt
index.txt

[abienvenido]

Hi, @ewalscargocare
You should also have SMTP AUTH ENABLED. - To go Office 365 Admin Center
Too, TLS 1.2.

I have made some changes. I hope it helps.

index.txt
Microsoft.txt
index2.txt

good luck!

[ewalscargocare]

Hi Abienvenido, thank you for your efforts! Unfortunately, that didn't help. I applied your changes and now I receive the following message:

Sorry, but what do you mean with SMTP AUTH ENABLED. - To go Office 365 Admin Center?

[abienvenido]

Hi Abienvenido, thank you for your efforts! Unfortunately, that didn't help. I applied your changes and now I receive the following message:

Sorry, but what do you mean with SMTP AUTH ENABLED. - To go Office 365 Admin Center?

Ok, this is good.

We are going to see two things:
1- Blue portal. Resource permissions
2 - See if we have enabled the SMTO AUTH of the account you are using.

Now you need to configure in Azure the permissions to the resource that your application requests.

In portal Azure, you have the next information

https://portal.azure.com/

Delegated permissions or application permissions.

For your case, it is probably worth having delegated permissions on the resource that you request from the API. if you use

For example.

if you use app permissions, an admin in your organization may need to pre-approve it for you in Azure.

https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/authenticated-client-smtp-submission

https://admin.microsoft.com/

[ewalscargocare]

Thanks again for the info Abienvenido! This is how the permissions are configured:

Can you confirm this is correct?

As for the SMTP auth, I asked my collegue to verify this now.

[decomplexity]

Re SMTP AUTH:
Log on to office.com as the user who registered the app in AAD
Select Admin
Select Users
Select Active users
Select the user (click the name, not the tick-box)
In the RHS screen panel, select the Mail tab
Select manage email apps
Make sure SMTP Auth is ticked

Re Abienvenido’s suggestions and your further comments:

• PHPMailer SMTP needs the SMTP.Send scope and permission. This is only available in the Graph API and (although the API resource is not listed by AAD) the Outlook REST API.

• it is unclear why AAD permissions are shown for both Graph and Office 365 Exchange resource APIs. It is valid, but the scopes used by the app can refer to only one resource API. Using a scope of e,g, “https://outlook.office.com/SMTP.Send https://graph.microsoft.com/Mail.Read” (or just https://outlook.office.com/SMTP.Send Mail.Read which is equivalent since the default is Graph) will give a ‘mixed resource’ error when you try to create a token.

• IMAP.AccessAsApp permission is not relevant, as PHPMailer is ‘send only’ and IMAP doesn’t ‘send’

[ewalscargocare]

Thanks Decomplexity! I will further test and come back with feedback asap.

[abienvenido]

Hi, @ewalscargocare

In Microsoft graph, SMTP.Send.

Too, you can see this:

TOKEN JWS

echo 'token jwt', $token = $provider->getAccessToken(
'authorization_code',
[
'code' => $_GET['code']
]
);
instead of using echo
error_log("TOKEN JWT: \n", 3, "/var/www/logs/php-token-jwt.log");

and see in
https://jwt.ms/

y with PHPMailer, what you have built:

$_SESSION['token'] = $token->getRefreshToken();

[ewalscargocare]

I was able to authenticate! SMTP auth was already activated, so I didn't need to change anything for that.
The actual fix was adding these permissions to Azure:

I use these URLS:

https://login.microsoftonline.com/82b7989c-1a0e-454b-9ed4-6ddeaa59ff7c/oauth2/v2.0/authorize
https://login.microsoftonline.com/82b7989c-1a0e-454b-9ed4-6ddeaa59ff7c/oauth2/v2.0/token

Scope:

offline_access https://outlook.office.com/SMTP.Send

Hope this helps other people!

[Synchro]

It would be awesome if you could summarise this in a wiki page.

[decomplexity]

Steve Maguire’s stevenmaguire/oauth2-microsoft provider was written before the V2 authorization and token endpoints were a common option. Jan Hajek’s thenetworg/oauth2-azure provider is up-to-date and includes them.
Note also that the Outlook REST V2.0 resource API, which has latterly been the bedrock of PHPMailer’s Exchange access, is scheduled to be decommissioned in November.
Its replacement, Graph V1.0, currently has SMTP authentication problems. It successfully produces an Access token, but the SMTP.Send validation in Exchange doesn’t like it.
I have discussed this with AAD support who have passed it to the Graph team. They are investigating and have agreed to come back to me w.c. 3rd October.

[abienvenido]

Hello @decomplexity

Do you have any new news about this issue from Microsoft?

Thank you very much!

[decomplexity]

I have discussed this at some length with MSFT's Graph team and raised issues that apparently need specialist input. One quirk I presented is that if a scope parameter URI is omitted, it appears not always to default to https://graph.microsoft.com and that the order of scopes specified in a scope statement apparently matters, both apparently contrary to MSFT documentation and sufficient to cause either successful obtaining of a refresh token or successful SMTP authentication to fail because a scope parameter is pointing at the wrong resource API.
I will keep this thread updated when I get some definitive further reply from them, but - counterintuitively - it seems that a scope parameter of https://outlook.office.com/SMTP.Send 'corresponds' to an AAD Graph permission of SMTP.Send.
In the interim, I suggest anyone with SMTP authentication problems uses:

• a client scope statement of just "offline_access https://outlook.office.com/SMTP.Send"
• thenetworg/oauth2-azure provider and not stevenmaguire/oauth2-microsoft provider
• V2.0 authorisation and token endpoints (an option with thenetworg/oauth2-azure)

and after registering a client app with AAD, don't bother to manually specify any permissions, Graph or otherwise, but instead Accept the 'Permissions requested' prompt you will receive when you try to create a refresh token; this will add suitable Graph permissions to the User consent list (to check, see AAD Enterprise applications =>[select your app] => select Permissions =>select the User consent tab)

[abienvenido]

Hi, @decomplexity

I'll be happy to follow the progress.
Thank you very much for your work and sharing it.

best regards

[Synchro]

Hi everyone, I though you'd like to know that @greew has very kindly put together an excellent wiki article about setting up Azure. He's also written a shiny new League adapter for Azure, and an implementation tweak to use that package in the PHPMailer OAuth setup example code via this PR.

Since you all seem to be trying to get this working, please could you give this new solution a try and let us know if you run into any issues?

[matteo-cavalli]

Hi all, i'm working on phpmailer+oauth2+microsoft from some days, and i'm in struggle with this.
I was using stevenmaguire as provider, with many option, many tentatives, many errors, token unreadable on jwt.io, ecc.
So first ideas was "stop and %$& this". But i really need to configure oauth with phpmailer and a provider that can work. with stevenmaguire, smtp auth fail without log or reason, so i've read october decomplexity post so ok, i can try a new provider but i need some guides or assistance to implement it. i need to send a single mail when user make login (on other portal), and now only send mail task is missing., someone can help with also a partial code for this? thanks

[greew]

@matteo-cavalli Have you tried the wiki article??

[matteo-cavalli]

Hi, yes using azure as provider with a new and clean installation, every time that i make a request for get_oauth_token i can see many error ofr bas rquest uncaught function and so on. so in this case i've keep microsoft as provider but if you need some test or if you have another link i can test it.

[dudestefani]

hi, i facing the same issue descripted as @matteo-cavalli. when i try to get the refresh token in get an exception on the request, i will thrown in parse response function.
i followed the instructions in the wiki article, unfortunaltey i facing also the uncaught exception in IdentityProviderException: Unauthorized in greew/oauth2-azure-provider/src/Provider/Azure.php:89 by retrieving the refresh token.

A hint/idea something would be helpfull and made my day. thanks

[decomplexity]

dudestefani: if you are using Greew's oauth2-azure-provider, try temporarily changing line 14 (or thereabouts) in Azure.php from
protected $tenantId = '';
to
protected $tenantId = 'your Directory (tenant) ID';
where your Directory (tenant) ID is displayed in AAD near the top of the central block of App registrations = > select your app
It is normally an 8char-4char-4char-4char-12char code

[dudestefani]

@decomplexity thanks for the tip, but unfortunately nothing change. same behavior/issue as before and yes its Greew's oauth2-azure-provider

[matteo-cavalli]

Same for me as per @dudestefani , same error. i've tried to change file Azure.php but error is the same, with Greew's provider

[greew]

@dudestefani @matteo-cavalli

In the last few lines of azure_xoauth2.php, try changing from

//send the message, check for errors
if (!$mail->send()) {
    echo 'Mailer Error: ' . $mail->ErrorInfo;
} else {
    echo 'Message sent!';
}

to

try {
    $status = $mail->send();
} catch (IdentityProviderException $e) {
    print $e->getMessage();
    var_dump($e);
    exit(1);
}

//send the message, check for errors
if (!$status) {
    echo 'Mailer Error: ' . $mail->ErrorInfo;
} else {
    echo 'Message sent!';
}

And let us know the output of the exception.

Hopefully the exception message can tell us a bit more about, what went wrong :)

[dudestefani]

@greew i am not sure it helps, cause my issue is more in retrieving the refresh_token. first of all i need the token, then i can try so send a mail or did i get something wrong?

[greew]

@dudestefani

Sorry - my bad. I had forgot the context.

Let's try the same in the get_oauth_token.php file:

Replace

    $token = $provider->getAccessToken(
        'authorization_code',
        [
            'code' => $_GET['code']
        ]
    );

with

    try {
        $token = $provider->getAccessToken(
            'authorization_code',
            [
                'code' => $_GET['code']
            ]
        );
    } catch (IdentityProviderException $e) {
        print "<pre>";
        echo "An error occured" . PHP_EOL;
        echo "Exception message: {$e->getMessage()}" . PHP_EOL;
        echo "Response: {$e->getResponseBody()}";
        die;
    }

and check the exception message for more info :)