What Is 365Inspect?

365Inspect is a PowerShell utility that automatically audits an administrator's Microsoft Office 365* configuration settings, detects insecure configurations, and produces a graphical report describing any perceived security flaws or shortcomings in the Office 365 environment. It also provides power users the ability to easily expand the amount of insecure configurations it can detect.

In essence, 365Inspect is a vulnerability scanner for Microsoft Office 365.

▪ *Office 365 has been referred to by a host of names in its lifetime, including Office 365 and Microsoft 365. In this article, I'm going to refer to it as Office 365 or O365.

Origins and Purpose

I used to work as an ethical hacker at an information security firm called Soteria Security. For years, several experienced blue teamers and incident responders in Soteria had observed that real-world attacks involving Microsoft Office 365 were on the rise. The techniques used by adversaries, they reasoned, usually exploited basic security flaws in the victims' Office 365 configurations. Yet the victims often were not aware that these flaws existed, or that Office 365 security was even an area they should be concerned about.

This led those blue teamers to dream up a utility: a scanner-like tool that would automatically test an Office 365 environment against a list of secure best practices, and advise the user by detailing a list of areas where their configuration fell short. This conceptual tool should help the O365 operator transition from knowing relatively little about O365 security, to immediately having a snapshot of how secure (or insecure) their O365 environment is and understanding why their settings may be undesirable.

Since I had programming experience, and was a known proponent of automation and offensive security technique R&D, it was posed to me that perhaps I could author something that fit the bill. I thought it was a fantastic idea at first, and only grew more convinced as I researched and developed the experiment that eventually became 365Inspect. Some thoughts about the current global security atmosphere that influenced the direction I went in, and my feeling that the project would be at least a modest success:

• There were increasing public reports about Office 365 attacks and public brainstorming about O365 security in the years leading up to 365Inspect's development.
• Microsoft's various O365 for PowerShell libraries offered a programmatic means to interact with Office 365. PowerShell is also easily redistributable and (in theory) rapid to develop in. Those qualities made it ideal for creating a modular open-source scanning tool.
• There was no similar tool available. Microsoft develops such a tool internally, but does not publicize the source code of this tool and charges customers thousands of dollars or an expensive subscription fee to run it against their Office 365 environment. Therefore there was a dangerous complacency in the Office 365 security solution space that I felt I might be able to disrupt, to the benefit of end users who previously had to make due without many free O365 security options.
• Although O365 and its PowerShell libraries existed, resources for in-depth Office 365 security related development and Office 365 automation were subpar. Documentation was scattered and lacking, to the point that several websites dedicated to properly enumerating the “undocumented” features of PowerShell/O365 had sprung up to fill the void in official sources*. The open source community writ large had seemingly been dissuaded from authoring a scanner of this nature. Doing so involved not only learning how to describe dozens upon dozens of Office 365 security flaws in plain English, but using Microsoft's incomplete and oft-misleading documentation and a whole lot of trial and error to identify the right way to programatically capture those flaws. If I could wade through all of these obstacles alone, and capture what I found in a lasting and organized way, I might save many people the trouble of having to wade that deep again.

▪ *This lack of documentation and “official” assistance reminded me of another Microsoft situation from the past. In the early 2000s, Microsoft had long held that they would not fully document certain internal features of the Windows Operating System API. This sometimes increased the difficulty of developing drivers or certain user-mode Windows programs. The software reverse engineering community took matters into their own hands by documenting the features themselves. Some lessons, it would seem, are never learned.

Development

Day by day, I did just that: wade through obstacles. I became best friends with an intentionally misconfigured O365 deployment that was used as a testbed. I stared at a list of 50 or so O365 secure best practices, read technical documentation and other websites, and used gradual trial and error to author a snippet of PowerShell that encapsulated each idea. For the sake of simplicity and minimal overhead during development, I first accumulated these in a straight line console program that merely spit out row after row of text describing flaws in the user's Office 365 security settings. This type of program and output was good enough to reassure someone with an “engineer brain” that the concept and program logic worked as intended, but it was not very easily expandable or usuable by anyone else. It wasn't “user friendly,” so to speak.

I knew this going in, of course, and later split the straight line program into dozens of small modules. Each module was to contain one security test and the associated documentation. The decision to represent each vulnerability detection and its description as separate, easily-understood "modules" was an intentional one. This decoupling of detection logic from the main script and report template would enable developers and users to easily create additional 365Inspect detection logic. I then created the existing 365Inspect program harness to run each module sequentially as part of an “entire” vulnerability scan.

Finally, I wrote some rough templating code and provided 365Inspect the ability to produce a graphical report in HTML rather than purely console text output. This model is more easily understood by the average user, and enables anyone to extend 365Inspect with additional vulnerability checks and features. I used a cheap trick that I often use to prototype HTML templates quickly: design the template in Google Docs, then export it as HTML and insert content values as desired.

Today

As I've already linked several times, 365Inspect is freely available. It is actively maintained by Soteria. Which is good. I never would have claimed the original 365Inspect script as anything close to flawless software, it would have been maybe at “Beta” in traditional software development terms. Soteria's engineers seem to have fixed a few bugs and developed additional modules to expand 365Inspect's capabilities.

People still ask me, near-daily, for some type of advice or solution for Microsoft Office 365 security. That is partially the genesis of this post. I needed something to link those folks to, to explain what 365Inspect is, how it came to be, and what I have to do with any of it and why I'm offering advice in the first place.

How Did That Even Happen / Why The 365 Security "Gaps?"

It isn't exactly a secret in network security that mail servers, business email accounts, firewalls or filters with weak rule logic, legacy or broken authentication, and so on are targets for adversaries. So why was an entire wave of IT administrators and organizational leaders caught off guard when adversaries targeted these flaws in Office 365?

Part of the answer is that, although slowly growing in popularity, internal cloud security has never been quite as popular a topic as network and application security. However, a more sinister second part of the answer is that popular cloud solutions disguise these issues for the sake of user convenience and marketability, and this disguise can convince people to utilize technologies that they don't fully understand.

The cloud is sold to individuals and businesses using the same tempting siren song as most technology sales through history: "It eases your duties. It has less of a physical footprint and learning curve than other technologies. It's more intuitive. It's easier to set up." Cloud technology is selling a packaged promise that the operator can think less and know less about the underlying powerful technology, yet still leverage powerful technology to achieve their goals.

That does sound fun. I'm in favor of removing the technological impediments that sometimes prevent people from changing the world. And I'm all in favor of encouraging people to construct less complex systems. Systems that are less complex are often less likely to break, more provably secure, can see more widespread use, and have less potential for errors and misuse.

Yet the same way a physicist would be inherently suspicious of another physicist stating "I found a way to modify physical matter without expending any energy," a technologist or engineer should be inherently suspicious of statements like "we removed the complexity from this process or system" or even “we adequately shielded the user from this process or system.” Presenting a system with low complexity (and therefore low potential for horrendous error or difficulties during use) often requires one or more of:

• Identifying the specific types of problem the system is supposed to solve, and building the system to solve only those problems. A bike is less complex to construct and has less potential for dangerous social misuse than an amphibious airplane, but also can't solve quite the same array of transportation-related problems.
• Making assumptions about what parameters are acceptable to the future operators of the system for solving their problems, and baking these assumptions into the system. Take two cars. The core problem solved by both is "people want to drive on paved roads." If we further assume that "the average person seldom needs to drive on a paved road at faster than 120 miles per hour," and build that limitation into the car, we can construct a cheaper, more reliable car than a Ferrari or what have you, because we've placed a realistic limitation on usage. And, again, we've (admittedly slightly) reduced the potential for disastrous operational decisions. You can't flee the authorities and hit a toll booth at 200 miles per hour if your car doesn't present you the option of going 200 miles per hour.

Using design techniques like those to reduce the complexity and increase the security of a system requires time, effort, and sometimes significantly reduces the perceived usefulness of that system. It can make the system inflexible. It can make it unmarketable. Take, for example, the ability to create user accounts with varying levels of business permissions (such as “this person can read existing documents, but not create new ones.”) Designing cloud software with the ability to do this dramatically increases how complex and insecure that software can become, but if O365 didn't have that capability, a large subset of potential O365 customers would consider that too big of an operating restraint--like having a car that only goes 15 miles per hour.

Products like Office 365 are sold using a different tact. The complexity is hidden behind an interface that purports to shield the user from the complexity and reduce the requirement that they educate themselves or coordinate with others. Although Office 365 is incredibly complicated software that can be quite difficult and complex to securely deploy at scale, the presence of default values and an accessible graphical interface give people the idea that it's a quick and easy rollout with minimal risk. Many of the O365 deployments that were “easy rollouts,” and required minimal IT security knowledge and staff, outgrow their original footprints and, without proper administration, gain increasingly tangled mail transport and identity and access management configurations that adversaries eventually exploit.