Jupyter for security
The problem
Security analysts have over 40 tools and data sources that they use when responding to security incidents. One of these tools was a web app that allowed users to search different data sources by IP address.
My role was to update this web app. As I progressed in my user research, I learned analysts were overwhelmed by the number of tools they used. To address this, I decided to create a product that allows security analysts to search all their data sources from one place.
Role: Engineer, storyteller
Team size: 3
Tools: Python, JupyterHub, Docker, Azure Pipelines, YAML
Skills: Planning, survey writing, UX writing, technical writing, coding, presenting, threat modeling
Timeline: 8 months
Outcome: Product
My process
User research
In order to understand how to bring these security tools together, I first had to understand them individually. I spent the first portion of this project shadowing users and collecting survey responses. The data I collected showed me how the tools were used and how often. I wrote wiki articles for some of the more confusing tools.
Sample of survey data that shows how often tools are used by users. Tool names are redacted for privacy.
After my research, I created the user personas, value statements, and acceptance criteria for the project.
Value statement
As an analyst, I want the ability to query multiple data sources from a central location so that I can quickly find the data required to solve and close incidents without opening multiple windows.
Development and documentation
Python modules
We developed individual Python modules for the tools and used ipywidgets to create simple user interfaces for analysts less familiar with raw code.
Interactive documentation
Using the Python modules, we created interactive documentation that allowed analysts to use the tools and respond to incidents in Jupyter.
Hub hosting
We hosted the service using JupyterHub and deployed the code from our repository to an Azure virtual machine using YAML pipelines. JupyterHub spawns Docker containers for each user. This allowed analysts the freedom to create and modify their own content without risk of being overwritten or requiring any additional setup.
Architecture diagram of JupyterHub
The product
The resulting product is a collection of modular reconnaissance tools that leverage key security datastores, allowing analysts to perform faster and deeper investigations in one portal.
We collected survey data from usability testing and learned average investigation time was reduced from 52 hours to 25 minutes on average.
I wrote and presented the story of our product to the Microsoft CEO to highlight our success in improving the security analyst experience and the power of combining Microsoft tools with open source technology.
Image of the JupyterHub interface. Sensitive information has been redacted.