Socrata

Transparency and security are no longer mutually exclusive.

Socrata uses Chef to keep government wide open, and secure data locked down.
"Chef helped lower the stress of this job. Before Chef, I spent a significant amount of time tied to a pager to ensure our datacenters were running smoothly. Our team still carries the pager 24x7, but it does not ring as often. We have dramatically increased the level of automation and auditability when we deploy new capacity and that gives us peace of mind. We can focus more on value creation for our customers, and the growth of our company." Paul Paradise, Operations Engineer

The Challenge

Socrata is the leading developer and provider of Open Data Services, a category of cloud-based Web 2.0 solutions that enable federal, state, and local governments to dramatically improve the reach, usability and social utility of their public information assets. As a data-publishing and consumption platform at those levels, Socrata manages multiple data centers that handle its Open Data cloud operations including more than 30 customers spanning all levels of government. In order to achieve compliance with the Federal Information Security Management Act (FISMA) of 2002 more quickly, Socrata needed an auditable, automated and secure system to scale configuration management and compliance with policies.

The Solution

Socrata found its solution in the power and simplicity of Chef's Chef (OC). Operating at increasing scale, manual configuration of servers and nodes becomes impossible, especially with the sophisticated Open Data Platform infrastructure, needed to manage the frictionless flow of Big Data that Socrata develops and operates. Chef delivered increased speed, efficiency, security, auditability and automation, enabling Socrata to easily and quickly obtain FISMA ATOs, reducing compliance time from months to weeks, and allowing Socrata engineers to focus more on innovation and less on maintenance – major benefits for its customers.

The Results

  • Increased agility and reliability to efficiently manage platform configuration
  • Significantly reduced time to secure authorizations to operate (ATOs)
  • Significantly upgraded Socrata's ability to deploy more capacity, virtually, on demand
  • Repeatable, fully automated infrastructure

Socrata's Recipe for Success

Socrata is a developer and provider of Open Data Services, a category of cloud-based Web 2.0 solutions that enables federal, state and local governments to dramatically improve the reach, usability and social utility of their public information assets. Socrata manages multiple data centers that handle its Open Data cloud operations including more than 30 customers spanning all levels of government.

For its customers to be in full compliance with the Federal Information Security Management Act of 2002 (FISMA) and to receive authorizations to operate (ATO) for the orgnizations it serves, Socrata needed a more auditable and secure system to successfully navigate within various security impact categorizations that are established at the governmental level (with federal ATOs at the highest level). Those impacts are then mapped to a set of stringent security controls published by the National Institute of Standards and Technology. Configuration management controls and their documentation became paramount for Socrata.

As Socrata’s sophisticated cloud-based infrastructure grew, manual configuration and maintenance became nearly impossible. In addition, Socrata was relying on a frequently outdated wiki-page to keep a running tab of system organizational steps and documentation.

There was no real automation to speak of,” says Paul Paradise, Socrata lead operations engineer.

The company looked to Chef’s Chef to provide the solution. “Prior to using Chef, we had had a patchwork of automation scripts that were becoming increasingly difficult to maintain with the rapid pace of growth in customer launches and the volume of data under management on our platform.”

“Chef enables us to securely scale our configuration management automation as a virtual on-demand capability, which saves us weeks of operational time and eliminates unnecessary infrastructure,” said Paradise. “It allows Socrata to focus our resources on helping customers turn their data into a strategic asset at a low cost of operation, while we manage the secure, redundant infrastructure needed to accommodate growing consumption of their data.”

For its new Chicago datacenter, Paradise was already codifying server set-up rules in Chef, and adding a new level of automation. Adding the second datacenter doubled Socrata’s number of machines, which could have been an onerous manual undertaking. “The time investment we made in writing those initial Chef recipes fundamentally changed our operations and freed up time to work on strategic activity,” notes Paradise. “I could go to Chicago, spend one day getting the physical hardware installed correctly and the next day basically boot everything up. By the third day, I was done. Most people don’t manage to get an entire datacenter set up in two and a half days. Without Chef and Chef, it might have taken two weeks.”