There is one Chef Server, and it is Open Source

Chef is used by companies of all shapes and sizes, from tiny startups to the largest companies in the world, to create businesses where infrastructure moves as fast as software. One thing that binds all these different companies together, regardless of size, is this critical fact:

The people who build and manage infrastructure are the people who best understand the unique requirements that make their businesses special.
That means that, when it comes to deciding how infrastructure automation should work, in the end, the people who run the business are the experts. I may be great at using Chef to tackle all sorts of interesting problems, but how to apply that knowledge depends entirely on the unique requirements of that business.

It is this reality that is leading most of us to build the future of our businesses on Open Source software. We just aren’t interested in waiting for, or pretending, that a closed source vendor understands our needs better than we do. We know it isn’t true.

Read more ›

Webinar recording: Chef for Containers

Tom Duffield gave a webinar on managing containers, including Docker, with Chef earlier today. Tom’s presentation includes a great overview of Docker and insights into why and how you can use Chef to improve your container workflow.

If you weren’t able to see the webinar live, here’s a recording:

Read more ›

Chef Releases Chef 12 to Power DevOps Practices in the Enterprise

Company Converges to Single Source Code Base and Introduces Tiered Subscription Model to Deliver Web-Scale IT in the Enterprise

IT Automation Platform Delivers Comprehensive Integration with Windows PowerShell DSC, VMware vCloud Air, and Amazon Web Services

SEATTLE – September 8, 2014 – Chef, the leader in web-scale IT automation, today released Chef 12, the next generation of its market-leading IT automation platform. Chef 12 delivers a single, open-source code base augmented by premium features, enabling maximum open-source innovation and delivering that value for commercial customers. Chef 12 includes new high availability, replication, and analytics capabilities.

Chef 12 also provides comprehensive integration with Windows PowerShell Desired State Configuration (DSC), VMware’s vSphere and vCloud Air, and Amazon Web Services (AWS). These integrations strengthen Chef’s position as the foremost platform for automating IT infrastructure and application delivery across Windows, Linux, and Unix, in the data center or on any combination of cloud platforms.

Read more ›

Chef Brings DevOps Platform and Practices to Windows and Microsoft Azure

New Chef Integration with Windows PowerShell Desired State Configuration Enables Users to Automate Workloads and Accelerate Application Delivery In Both Windows and Linux Environments

SEATTLE – September 8, 2014 –Chef, the leader in web-scale IT automation, today announced new integration with Windows PowerShell Desired State Configuration (DSC), enabling developers and operations teams to best implement DevOps practices on the journey to web speed and scale. Chef also today announced its next-generation IT automation platform, Chef 12 (see separate release), with a new comprehensive suite of enterprise features for high availability (HA), replication, and analytics.

Read more ›

Chef Delivers DevOps Automation for VMware vCloud Air

Today we announced interoperability with VMware vCloud Air, helping create a DevOps workflow that accelerates software and service delivery. Chef provides a single platform for automating heterogeneous systems in both virtualized and hybrid cloud environments. By delivering interoperability with VMware vSphere and VMware vCloud Air, Chef will enable VMware customers to more easily manage data center workloads and streamline migrations to the cloud.

Read on for the details of our new capabilities for VMware vCloud Air…

Read more ›

Release: Chef Client 11.16.0 & Ohai 7.4.0

Ohai Chefs,

Today we’ve released Chef Client 11.16.0 and Ohai 7.4.0. The releases contain the following changes:

Chef Client 11.16.0

  • MacOS: Fix dscl user provider to be able to manage home and password at the same time
  • Support for  PowerShell Desired State Configuration (DSC) on Windows
  • Update to OpenSSL 1.0.0n on Windows and 1.0.1i on all other platforms
Ohai 7.4.0
  • Ohai PowerShell plugin

OpenSSL Security Update

This release includes OpenSSL 1.0.1i (1.0.0n on Windows) in response to the 2014-08-06 OpenSSL Security Advisory. You should update to this version if these vulnerabilities affect you.

Special thanks to Jesse Hu for bringing to our attention that the last release of the client didn’t get the updates. Jesse, you’re this release’s MVP for watching out for all of us.

Our thanks go out to:

Read more ›

Announcing Chef Server High Availability and Replication

Today I’m thrilled to announce two new add-ons for Chef Server 12: Chef Server High Availability and Chef Server Replication. These two features are among the most-frequently requested product enhancements and allow customers to geographically distribute highly-available Chef server clusters while maintaining a single view for Chef content – cookbooks, roles, environments, and data bags. These, and other add-ons, can be obtained from the Chef download site.

Chef Server High Availability

In Enterprise Chef 11 and prior, high-availability Chef server clusters could only be reliably built on bare metal, using a shared block device with a technology known as DRBD (distributed replicated block device). DRBD now ships with the base Chef Server 12 product, and today we are introducing a new add-on that will support additional deployment scenarios, such as those in the cloud.

At launch, Chef Server HA supports Amazon Web Services (AWS), because it brings the necessary infrastructure primitives to the table: a block device that can be detached and reattached to the backend Chef servers, as well as a floating (“elastic”) IP. We intend to expand our high-availability support to other popular clustering technologies, like RedHat Cluster Manager, and other clouds, whether public or private, that provide similar kinds of infrastructure primitives. The following diagram illustrates the deployment scenario of Chef Server High Availability in AWS.

Chef Server HA in AWS diagram

There is no high-availability between regions. It is generally considered poor system design to attempt synchronous operations over high-latency links. For that, you likely want to implement Chef Server Replication.

For more information, please see the Chef Server High Availability documentation or skip directly to the section on AWS..

Chef Server Replication

If you have multiple Chef servers in multiple regions (they can be high-availability Chef servers as well), you may be faced with the problem of keeping the content in these servers consistent. That’s where our new Chef Server replication feature comes in. Using this add-on, you designate one of your Chef servers as the primary, and one or more other Chef servers as replicas. Each replica will periodically awaken and synchronize any changed content from the primary: cookbooks, roles, environments, and data bags. Replication can be configured on a per-organization basis. It is also consistent across network partitions.

A typical deployment scenario is to place the primary in one geographic region, and replicas in other geographic regions, as this diagram illustrates.

Chef Server Replication deployment diagram.

For more information on Chef Server Replication, please see Chef Server Replication documentation.

Conclusion

We’re introducing Chef high availability and replication today to meet the demands of larger enterprises that want to build highly-available, multi-region Chef server deployments. We’d be delighted if you’d try out these add-ons, and we welcome your feedback.

Chef Client 11.16.0 gets into PowerShell DSC

Ohai Chefs. Today’s release of Chef Client 11.16.0 marks the inclusion of PowerShell Desired State Configuration (DSC) support into Chef Client for Windows. DSC is a powerful configuration management platform built into PowerShell 4.0, and now you can use it with Chef!

To try it out, just configure a system with Chef Client 11.16.0 or later and target it with a recipe that uses the new dsc_script resource, which you can learn about on our documentation site.

Like Chef, DSC exposes resources to configure systems. The rest of this post gives details on how to use Chef’s new dsc_script resource to gain access to all of DSC’s resources from your recipes, and also discusses where we’re headed with DSC in the future.

Read more ›

Announcing Chef Server 12 Release Candidate

Today we’re pleased to announce the public availability of the first Chef Server 12 Release Candidate. This release brings the differentiating features of Enterprise Chef, namely multi-tenancy and role-based access control, into Open Source Chef.

What’s New

Chef Server 12 brings a host of improvements that will be welcomed by existing Open Source and Enterprise customers alike. Here are just a few:

All PostgreSQL, All the Time

Eighteen months ago, we released Chef 11 to the world. It was awesome, and it increased the scalability of the Chef Server to new levels. At the time, we had just rewritten the entire core of the API in Erlang and had migrated all of the data that the server stores from CouchDB to PostgreSQL. We soon followed up with Enterprise Chef 11, which brought this same scalability to our commercial customers.

Over the past year, we’ve been working tirelessly on removing the final bits of the CouchDB-backed Ruby API from the multi-tenancy features of the Chef Server, and we’re pleased to announce that, with this release, we’ve completed that task.

Advancing the Chef Server Platform

With the removal of CouchDB and the consolidation of the Chef Server API into one codebase, we’ve been able to reduce the complexity and the footprint of the Chef Server platform. This starts with a 20% reduction in package size and a 25% reduction in the number of active services running on the Chef Server.

Building on top of this, we now have a fresh, consistent, and open-source codebase that will help us accelerate new feature development around the ways that you interact with your Chef Server. Some of our most-requested features like external authentication, external groups via LDAP, and templated organization policy can be built atop Erlang and PostgreSQL — a technology stack that we’re heavily invested in at Chef and one that won’t be going away anytime soon.

Search Improvements – Solr 4 Brings the Speed

Chef uses the Apache Solr search server to handle the search-based discovery for your Chef infrastructure. The version of Solr (1.4) that ships with the Chef 11 Servers was released in November 2009. In technology terms, that’s ancient. Over the past 5 years, the Solr team has shipped some amazing features and improvements to their product, and we’re pleased to announce that the Chef community will now get to experience those.

Since its inception, one of the most frequently requested improvements to Chef Server has been to reduce the amount of time that it takes for a saved object to become searchable. In Chef 11 and earlier, that latency was anywhere between one and sixty seconds. If you had a heavily loaded Chef Server, then it may take even longer while the asynchronous process sending your data to Solr was catching up to the request load. At the worst of times, Hosted Chef would experience search latencies longer than 10 minutes.

Solr 4 has allowed us to solve this problem. We’ve been running this version of Solr in Hosted Chef for a few months now, and the average search latency is just over one second.

Instant Access to Premium Features

For the first time ever, every user of Chef will have instant access to try out any of the premium features provided by Chef Software simply by running

chef-server-ctl install $premium-feature-name

Those premium features include:

  • Analytics Platform: Get visibility into your Chef servers, verify compliance and keep up with changes, all with the Chef analytics platform.
  • Management Console: Use the web-based management console to manage RBAC, edit and delete nodes, and reset private keys. Keep up to date with what’s happening during chef client runs across an entire organization or on specific nodes.
  • Reporting: Capture and visualize what happens during the execution of chef-client runs across all of your Chef-managed infrastructure.
  • High Availability: Ensure that your Chef service is uninterrupted within your data center or AWS region, even if a Chef server fails.
  • Replication: Maintain a single worldview across multiple locations and ensure consistency across your network, no matter how many data centers or cloud availability zones you use in your enterprise.

All of these features are free for installs of fewer than 25 nodes, and come with a 30 day free trial for larger installs.

Chef Identity – Run Your Own Supermarket

Chef 12 now ships with Chef Identity included, allowing external applications like Supermarket and Chef Analytics to authenticate with Chef Server. Take a look at the blog post for Supermarket for instructions on setting this up.

Where to Get It

To download the Chef Server 12 RC, visit http://downloads.getchef.com/chef-server and select your operating system.

Once you’ve got the package downloaded, follow the instructions at http://docs.getchef.com/server/.

If you’re upgrading from a previous release, the following versions are currently supported: * Enterprise Chef 11.1.8 and later * Open Source Chef Server 11.1.0 and later

Known Issues

The following issues are known and will be addressed before the final release of Chef Server 12:

  • Secure LDAP: For existing Enterprise Chef users of external authentication over LDAPS, this feature is currently broken.
  • Push Jobs Feature: The Push Jobs package does not currently install on Chef Server 12. We will be updating the package metadata before the final release to address this issue.

What’s Next

Today’s release is the first release candidate of Chef Server 12. Because this is the first time we’ve been able to speak publicly about Chef 12, the amount of feedback that we’ve been able to gather from you, our community and customers, has been limited. We’re asking that you kick the tires on this release like you would any new release. You can even drive it like a rental car if you’d like :). Whether you’re a new user or upgrading from an existing install, we’re interested in hearing about your experiences. If you run into something unexpected, we encourage you to do one of two things:

Our support and engineering teams will be monitoring these channels extensively over the next days and weeks, taking in your feedback and preparing Chef 12 for final release.

On behalf of the entire team at CHEF, Enjoy!

Release: Enterprise Chef 11.2.1

Enterprise Chef 11.2.1 is a critical bug-fix release for customers who installed Enterprise Chef 11.2.0. It corrects a single defect experienced by customers who upgraded from earlier releases.

Bug Fixes:

  • Fixes an issue where private-chef was being changed to private_chef unexectedly in upstart/runit configuration files

Notes:

If you upgrade from an earlier release of EC, your servers may now have two runit processes configured in upstart
  1. /etc/init/private-chef-runsvdir.conf
  2. /etc/init/private_chef-runsvdir.conf
The second one is incorrect, introduced by the aforementioned issue in EC 11.2.0. In this condition, you will see two runsvdir processes running with many errors:

ps:

root       924     1  0 05:20 ?        00:00:00 runsvdir -P /opt/opscode/service log: /lock: temporary failure runsv ocid: fatal: unable to lock supervise/lock: temporary failure runsv couchdb: fatal: unable to lock supervise/lock: temporary failure runsv bookshelf: fatal: unable to lock supervise/lock: temporary failure runsv postgresql: fatal: unable to lock supervise/lock: temporary failure runsv opscode-certificate: fatal: unable to lock supervise/lock: temporary failure 
root       926     1  0 05:20 ?        00:00:00 runsvdir -P /opt/opscode/service log: ry failure runsv opscode-expander: fatal: unable to lock supervise/lock: temporary failure runsv opscode-solr: fatal: unable to lock supervise/lock: temporary failure runsv rabbitmq: fatal: unable to lock supervise/lock: temporary failure runsv ocbifrost: fatal: unable to lock supervise/lock: temporary failure runsv opscode-chef-mover: fatal: unable to lock supervise/lock: temporary failure 

pstree:

Correcting the error:

HA

  1. on both the active/bootstrap and standby backend: remove the errant runsvdir config file
    root@backend1# rm -f /etc/init/private_chef-runsvdir.conf
    root@backend2# rm -f /etc/init/private_chef-runsvdir.conf
    
  2. On the standby (non-bootstrap) backend: reboot your server to clear all remaining orphaned processes and to restart runsvdir to a working state

    root@backend2# init 6

  3. On the standby backend: Verify that there is only a single runsvdir process and it is error-free (all dots)

    root@backend2# ps -ef |grep 'runsvdir -P /opt/opscode/service'
    root       921     1  0 05:35 ?        00:00:00 runsvdir -P /opt/opscode/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................

    root@backend2# private-chef-ctl ha-status [OK] keepalived HA services enabled. [OK] DRBD disk replication enabled. [OK] DRBD partition /dev/opscode/drbd found. [OK] DRBD device /dev/drbd0 found. [OK] cluster status = backup [OK] did not find VIP IP address and I am not master [OK] found VRRP communications interface eth0 [OK] my DRBD status is Connected/Secondary/UpToDate and I am not master [OK] my DRBD partition is not mounted and I am not master [OK] DRBD primary IP address pings [OK] DRBD secondary IP address pings [OK] bookshelf is not running, and I am not master. [OK] couchdb is not running, and I am not master. [OK] keepalived is running. [OK] nginx is not running, and I am not master. [OK] oc_bifrost is not running, and I am not master. [OK] oc_id is not running, and I am not master. [OK] opscode-account is not running, and I am not master. [OK] opscode-certificate is not running, and I am not master. [OK] opscode-erchef is not running, and I am not master. [OK] opscode-expander is not running, and I am not master. [OK] opscode-expander-reindexer is not running, and I am not master. [OK] opscode-org-creator is not running, and I am not master. [OK] opscode-solr is not running, and I am not master. [OK] opscode-webui is not running, and I am not master. [OK] postgresql is not running, and I am not master. [OK] rabbitmq is not running, and I am not master. [OK] redis_lb is not running, and I am not master.

    [OK] all checks passed.

  4. on the active/bootstrap backend: trigger a failover and then reboot
    root@backend1# private-chef-ctl stop keepalived
    ok: down: keepalived: 1s, normally up
    root@backend1# sleep 30
    root@backend1# init 6
    
  5. on the bootstrap (now standby backend): Verify that there is only a single runsvdir process and it is error-free (all dots)
    root@backend1# ps -ef |grep 'runsvdir -P /opt/opscode/service'
    root       921     1  0 05:35 ?        00:00:00 runsvdir -P /opt/opscode/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................
    
  6. On the active (non-bootstrap) backend, trigger another failover back to the bootstrap backend
    root@backend2# private-chef-ctl restart keepalived
  7. Test your now-active bootstrap backend to ensure full functionality (note: you may need to point your api_fqdn address at localhost using the server’s /etc/hosts file
    root@backend1# private-chef-ctl ha-status
    [OK] keepalived HA services enabled.
    [OK] DRBD disk replication enabled.
    [OK] DRBD partition /dev/opscode/drbd found.
    [OK] DRBD device /dev/drbd0 found.
    [OK] cluster status = master
    [OK] found VIP IP address and I am master
    [OK] found VRRP communications interface eth0
    [OK] my DRBD status is Connected/Primary/UpToDate and I am master
    [OK] my DRBD partition is mounted and I am master
    [OK] DRBD primary IP address pings
    [OK] DRBD secondary IP address pings
    [OK] bookshelf is running correctly, and I am master.
    [OK] couchdb is running correctly, and I am master.
    [OK] keepalived is running.
    [OK] nginx is running correctly, and I am master.
    [OK] oc_bifrost is running correctly, and I am master.
    [OK] oc_id is running correctly, and I am master.
    [OK] opscode-account is running correctly, and I am master.
    [OK] opscode-certificate is running correctly, and I am master.
    [OK] opscode-chef-mover is running.
    [OK] opscode-erchef is running correctly, and I am master.
    [OK] opscode-expander is running correctly, and I am master.
    [OK] opscode-expander-reindexer is running correctly, and I am master.
    [OK] opscode-org-creator is running correctly, and I am master.
    [OK] opscode-solr is running correctly, and I am master.
    [OK] opscode-webui is running correctly, and I am master.
    [OK] postgresql is running correctly, and I am master.
    [OK] rabbitmq is running correctly, and I am master.
    [OK] redis_lb is running correctly, and I am master.

    [OK] all checks passed.

    root@backend1# private-chef-ctl test ... Finished in 1 minute 23.67 seconds 116 examples, 0 failures, 3 pending

    • Note: pending errors are OK
    • Note: This command may fail on the first attempt after a fail-over, please contact support if it continues to fail.
  8. On your frontends, follow the Standalone procedure as detailed below
  9. Upgrade following the normal procedure to Enterprise Chef 11.2.1

Standalone

  1. stop the errant runsvdir process:
    # initctl status private_chef-runsvdir
    private_chef-runsvdir start/running, process 926

    # initctl stop private_chef-runsvdir private_chef-runsvdir stop/waiting

  2. remove the errant runsvdir config file
    # rm -f /etc/init/private_chef-runsvdir.conf
    
  3. stop all private-chef services
    # private-chef-ctl stop
    
  4. reboot your server to clear all remaining orphaned processes and to restart runsvdir to a working state.
  5. Verify that there is only a single runsvdir process and it is error-free (all dots)
    # ps -ef |grep 'runsvdir -P /opt/opscode/service'
    root       921     1  0 05:35 ?        00:00:00 runsvdir -P /opt/opscode/service log: ...........................................................................................................................................................................................................................................................................................................................................................................................................
    
  6. Test your system to ensure full functionality (note: you may need to point your api_fqdn address at localhost using the server’s /etc/hosts file
    # private-chef-ctl test
    ...
    Finished in 1 minute 23.67 seconds
    116 examples, 0 failures, 3 pending
    
    Note: pending errors are OK

Archives