Office 365 Recommended Configuration Analyzer

In this article, you will learn how to improve security in Exchange Online – Office 365. We will use the Microsoft Defender for Office 365 Recommended Configuration Analyzer (ORCA) tool and create an ORCA report. We will also look at the configuration analyzer in Microsoft 365 Defender portal.

When you have Exchange Online or Microsoft 365, most think they don’t have to do anything anymore. Everything is in the cloud, and Microsoft will do the rest. We are sorry to tell you, but that’s completely wrong. You need to identify the issues in your current configuration and improve them for better security.

Office 365 Recommended Configuration Analyzer (ORCA)

ORCA is a report you can run in your Microsoft 365 environment, highlighting known configuration issues and improvements that can impact your experience with Microsoft Defender for Office 365 (formerly Office 365 Advanced Threat Protection).

The configuration analyzer analyzes the following types of policies:

• Exchange Online Protection (EOP) policies: This includes Microsoft 365 organizations with Exchange Online mailboxes and standalone EOP organizations without Exchange Online mailboxes.

• Microsoft Defender for Office 365 policies: This includes organizations with Microsoft 365 E5 or Defender for Office 365 add-on subscriptions.

You can run the ORCA report without Microsoft Defender for Office 365, but there will be fewer checks. Always verify the latest ORCA version from the PowerShell Gallery.

Install ORCA module

Start Windows PowerShell as administrator.

Run the Install-Module ORCA cmdlet to install the ORCA PowerShell module.

Install-Module ORCA -Force

Verify that you successfully installed the ORCA module with the Get-InstalledModule cmdlet.

Get-InstalledModule -Name ORCA | ft -AutoSize

The output shows.

Version Name Repository Description
------- ---- ---------- -----------
2.8.1   ORCA PSGallery  The Microsoft Defender for Office 365 Recommended Configuration Analyzer (ORCA) Modul

Get ORCA report

Run Get-ORCAReport cmdlet.

Get-ORCAReport

Note: If you don’t have the Exchange Online PowerShell module installed, it will ask you to install it.

A window will appear and ask you for your Global administrator credentials.

Let it analyze the tenant and go through the recommendation checks.

08/03/2023 13:10:38 Performing ORCA Version check...
08/03/2023 13:10:44 Connecting to Exchange Online (Modern Module)..
08/03/2023 13:10:48 Getting Anti-Spam Settings
08/03/2023 13:10:48 Getting Tenant Settings
08/03/2023 13:10:49 Getting MDO Preset Policy Settings
08/03/2023 13:10:49 Getting Protection Alerts
08/03/2023 13:10:51 Getting EOP Preset Policy Settings
08/03/2023 13:10:52 Getting Quarantine Policy Settings
08/03/2023 13:10:54 Getting Anti Phish Settings
08/03/2023 13:10:54 Getting Anti-Malware Settings
08/03/2023 13:10:54 Getting Transport Rules
08/03/2023 13:10:55 Getting MDO Policies
08/03/2023 13:10:55 Getting Accepted Domains
08/03/2023 13:10:55 Getting DKIM Configuration
08/03/2023 13:10:56 Getting Connectors
08/03/2023 13:10:56 Getting Outlook External Settings
08/03/2023 13:10:56 Getting MX Reports for all domains
08/03/2023 13:11:09 Determining applied policy states
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Domain Allowlist
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Spam Action
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Domain Allowlisting
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Allowed Senders
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - IP Allow Lists
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Safety Tips
08/03/2023 13:11:09 Skipping - Safety Tips - No longer part of Anti-Spam Policies
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Anti-Spam Policy Rules
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Phish Action
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - High Confidence Spam Action
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Bulk Action
08/03/2023 13:11:09 Analysis - Anti-Spam Policies - Mark Bulk as Spam
08/03/2023 13:11:10 Analysis - Anti-Spam Policies - High Confidence Phish Action
08/03/2023 13:11:10 Analysis - Anti-Spam Policies - Quarantine retention period
08/03/2023 13:11:10 Analysis - Anti-Spam Policies - Outbound spam filter policy settings
08/03/2023 13:11:10 Analysis - Anti-Spam Policies - Advanced Spam Filter (ASF)
08/03/2023 13:11:10 Analysis - Anti-Spam Policies - Bulk Complaint Level
08/03/2023 13:11:10 Analysis - Connectors - Enhanced Filtering Configuration
08/03/2023 13:11:10 Analysis - Connectors - Domains
08/03/2023 13:11:10 Analysis - DKIM - DNS Records
08/03/2023 13:11:11 Analysis - DKIM - Signing Configuration
08/03/2023 13:11:11 Analysis - Malware Filter Policy - Common Attachment Type Filter
08/03/2023 13:11:11 Analysis - Malware Filter Policy - Malware Filter Policy Policy Rules
08/03/2023 13:11:11 Analysis - Malware Filter Policy - Internal Sender Notifications
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Alerts - Protection Alerts
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Policies - Anti-phishing trusted senders
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Policies - Safe Attachments Policy Rules
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Policies - Anti-phishing Rules
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Policies - Anti-phishing trusted domains
08/03/2023 13:11:11 Analysis - Microsoft Defender for Office 365 Policies - Safe Links Policy Rules
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - User Impersonation Action
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Intra-organization Safe Links
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Documents for Office clients
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Similar Users Safety Tips
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Mailbox Intelligence Protection
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Links protections for links in office documents
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Links protections for links in teams messages
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - First Contact Safety Tip
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Built-in Protection
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Links protections for links in email
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Anti-spoofing protection action
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Do not let users click through safe links
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Do not let users click through Safe Documents for Office clients
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Unauthenticated Sender (tagging)
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Mailbox Intelligence Enabled
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Links Synchronous URL detonation
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Similar Domains Safety Tips
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Links Tracking
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Domain Impersonation Action
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Unusual Characters Safety Tips
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe attachments unknown malware response
08/03/2023 13:11:12 Analysis - Microsoft Defender for Office 365 Policies - Safe Attachments SharePoint and Teams
08/03/2023 13:11:13 Analysis - Microsoft Defender for Office 365 Policies - Safe Links Allow Listing
08/03/2023 13:11:13 Analysis - Microsoft Defender for Office 365 Policies - Mailbox Intelligence Protection Action
08/03/2023 13:11:13 Analysis - Microsoft Defender for Office 365 Policies - Safe Attachments Allow listing
08/03/2023 13:11:13 Analysis - Microsoft Defender for Office 365 Policies - Anti-spoofing protection
08/03/2023 13:11:13 Analysis - Microsoft Defender for Office 365 Policies - Advanced Phishing Threshold Level
08/03/2023 13:11:13 Analysis - Outlook - External Tags
08/03/2023 13:11:13 Analysis - Quarantine Policies - End-user Spam notifications
08/03/2023 13:11:13 Analysis - SPF - SPF Records
08/03/2023 13:11:13 Analysis - Tenant Settings - Unified Audit Log
08/03/2023 13:11:13 Analysis - Transport Rules - Domain Allow Listing
08/03/2023 13:11:13 Analysis - Transport Rules - Domain Allow Listing
08/03/2023 13:11:13 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Phish
08/03/2023 13:11:13 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Malware
08/03/2023 13:11:13 Analysis - Zero Hour Autopurge - Supported filter policy action
08/03/2023 13:11:13 Analysis - Zero Hour Autopurge - Zero Hour Autopurge Enabled for Spam
08/03/2023 13:11:14 Generating Output
08/03/2023 13:11:14 Output - HTML
08/03/2023 13:11:20 Complete! Output is in C:\Users\administrator.EXOIP\AppData\Local\Microsoft\ORCA\ORCA-exoip365-202308031311.html

After the above checks, an HTML report is generated and exported to the AppData folder. In the next step, we will look at the ORCA HTML report.

ORCA report details

By default, the HTML report will open in your default browser. The first thing that we noticed is the red block at the top. You will see that if you don’t have Microsoft Defender for Office 365 in the tenant. If it’s not shown, it means you have it, and ORCA performed extra checks.

Note: Use Microsoft Defender for Office 365 in your Microsoft 365 tenant for maximum security.

This is how it looks when Microsoft Defender for Office 365 is available.

Scroll down to the Summary and check the sections that the ORCA report analyzed.

Scroll to one of the sections that need improvement. In this example, we will look at the Anti-Spam Policies.

The Outbound spam filter policy settings have 4 settings with a current value that is not recommended, and it recommends we improve it.

Under each section, you find more information about the recommended settings. Clicking the links will open the Microsoft technical documentation page. The last link will take you straight to the settings to configure, which is excellent.

In the next step, we will look at the Office 365 recommendation configuration analyzer, but this time in the Microsoft 365 Defender portal. That’s because Microsoft did add a configuration analyzer to it.

Microsoft 365 Defender configuration analyzer

The configuration analyzer can help identify issues in your current configuration and help improve your policies for better security.

Go to the configuration analyzer in Microsoft 365 Defender portal by following these steps:

1. Sign in to Microsoft 365 security center
2. Expand Email & collaboration
3. Go to Policies & rules > Threat policies
4. Click on Configuration analyzer

Standard recommendations

The standard recommendations show 18 recommendations. We can select each recommendation and click on Apply recommendation to improve the policy.

The Standard and Strict policy settings values used as baselines are described in Recommended settings for EOP and Microsoft Defender for Office 365 security.

Strict recommendations

The strict recommendations show a total of 21 recommendations.

It’s up to every organization if they want to improve the policies with strict or standard recommendations.

Configuration analyzer shows different recommendations

The ORCA report shows different recommendations than the configuration analyzer in the Microsoft 365 portal. Why is it not similar? That’s because ORCA is developed by Product Managers at Microsoft, and it’s not an official Microsoft utility. For official, in-product configuration analysis, use the Microsoft 365 Defender configuration analyzer in the portal.

Follow the last ORCA report recommendations, and you’re all set. Did the configuration analyzer tool recommendations help you improve your Microsoft 365 tenant?

Read more: Microsoft Exchange Server vulnerability check »

Conclusion

You learned how to check and configure Office 365 security recommendations with the configuration analyzer. The ORCA report will give you more recommendations than in the security portal. Also, the report will give easier access to the Microsoft technical documentation page.

I recommend using both configuration analyzers for now. Adopt the change through the Microsoft 365 Defender portal if you only want to push a button. If you want to adjust the setting, go through the steps outlined in the Microsoft technical documentation, which you will find in the ORCA report under each section.

Did you enjoy this article? You may also like Fix Winmail.dat attachment in Office 365. Don’t forget to follow us and share this article.