Chef 10.14.0 Released!

We are quite excited about getting this version of Chef released and into your hands. This release contains over seventy-five resolved issues from over thirty community members and some big new features from Opscode.

Whyrun

This version includes a new feature that we’ve talked about for a long time called “why-run.” This feature allows you to run chef on a system without the providers modifying the system. A key part of why-run is that it tells you why it makes the decision that it did, so you can reason about the current and proposed state of your system, hence the name.

The no-operation or no-op concept is initially simple to reason about: decide what you’re going to do, but don’t actually do it. However, in configuration management the resources (the things you manage) that you are managing have interdependencies. If you do not account for this in the system, the proposed changes will differ wildly from the actual changes as the system grows in complexity. It is easy to build a system that doesn’t account for this truth, but from our experience building large infrastructure we knew we needed to engineer a proper solution that would scale.

For example, it is easy to model what happens when you use the service resource to start a service. If the action is start and the service is not running, you would start the service. If the service is already running, there would be no need to start it.

But what about when we need to start a service that is installed from a package? We can’t check to see if a service is running without installing the package first. A simple no-op mode would tell us that it would install the package but couldn’t tell us about the service because it could not determine its state.

This example is intentionally easy to reason about; you know that the service would be started after the package is installed. But service actions usually trigger notifications on other resources so it is important to get this right to produce a useful no-op mode.

Given the use cases for a no-op mode, Opscode’s engineering team came back with a design that allowed for certain assumptions to be made in each resource. If the service resource cannot find the appropriate command to check the status of the service, Chef’s why-run mode will assume that the command would have been installed by a previous resource and the service would not be running. This of course isn’t guaranteed to be the case, but we believe this will greatly increase the usefulness of why-run over other possible designs.

Another area of concern is what to do with not_if and only_if attributes. Chef assumes these are commands or blocks that are safe to run in why-run mode. These conditionals are not designed to be used to change the state of the system, but rather to help facilitate idempotency for the resource itself. However, it is possible that you may be using them in a way that modifies the system state, so be aware of this.

The closer the current state of the system is to the desired state, the more useful why-run mode is going to be to you. Running a full run-list against a fresh system in why-run mode is not only unlikely to be completely correct, it is going to produce significant output which is probably less useful to you. Chef’s why-run mode is not a replacement for running your cookbooks in a test environment that mirrors production as closely as possible. Opscode uses test-kitchen on developer systems, an internal Openstack cloud and external cloud providers to do so.

Output formatters

Creating a useful output for Chef’s why-run also drove development of a new output formatter. In the past there have been debates for more or less informational logging at the default level. Some users only want Chef to output a message if it does something, others want to know what Chef is up to in the entire run. You could always specify what level of information was provided, but you can now specify how that information is presented using an output formatter. Combined with the log-level, this gives you the ability to tune output to your need.

# Short output with a single character per resource,
# followed by a summary
chef-client -Fminimal -lfatal 
# A verbose human readable format like rspec
chef-client -Fdoc -lfatal 

We’re planning on making the ‘doc’ output formatter the default client output in Chef 11, so please let us know what you think on the chef mailing list.

Error Inspectors

We’ve also got a new feature that captures specific errors from Chef and tells you why it believes the error occurred. For example an error raised while compiling a recipe is going to present a formatted output that shows you the line in the recipe that Chef believes is responsible.

Solaris Omnibus

We now have Omnibus packaging for Solaris 9, 10 and 11 on sparc, as well as Solaris 10 and 11 on x86.

MVP

Xabier de Zuazo has been making some epic contributions. More than once he has set out to fix a small bug and ended up doing a significant code refactor for the project. It’s great to see a community contributor doing such essential work. Congratulations Xabier, you’re this release’s MVP!

Notable changes

  • Fixes a minor security bug that allowed you to get the list of environments from the WebUI without credentials.
  • Supports long host names and user names by trying the new mixlib-authentication protocol against the server and falling back to the old one.
  • Omnibus builds now include ruby-shadow when applicable.
  • Subscribes now support the newer notification syntax.

Contributors

  • Chris Roberts made the daemonized server run garbage collection before sleeping and added the ability for the chef-client to fork for each run.
  • Phil Dibowitz refactored ‘knife cookbook upload -a’ to support batch uploading and added support for knife cookbook site install to use the current branch.
  • Mitsuru Yoshida fixed some of the unit tests on FreeBSD and improved the FreeBSD service provider.
  • Ken-ichi TANABE improved the FreeBSD service provider when the RC var doesn’t match the service name.
  • KC Braunschweig provided a patch to add a config value for a list of directories for knife exec to search.
  • Jason J. W. Williams supplied a base IPS package provider for Solaris and friends.
  • Ben Lindsey updated the deploy provider to allow specifying the number of releases to keep.
  • Will Maier added the ability to pass arguments to the execute resource’s command attribute as an array.
  • Brandon Adams provided a patch for downloading all cookbook segments at the start of the run.
  • Sean Escriva fixed the user resource when changing a user’s home directory.
  • Hui Hu added a return code for knife ssh when it fails.
  • Anna Gabutero fixed a bug in the mdadm provider related to inactive arrays.
  • Takeshi Kondo patched knife ssh to use cloud attributes if available by default.
  • Martha Greenberg improved the solaris group provider to work on newer versions.
  • Michael D. Ivey added support for manipulating environments in shef and made it so all clients could delete themselves.
  • Mike Conigliaro provided a patch to the debian packaging to fix a logrotate bug.
  • Nathen Harvey improved the initial skeleton cookbook that knife creates.
  • David Amian fixed a bug that was preventing nodes from being deleted in the WebUI using firefox.
  • Xabier de Zuazo added support for month and day names to the cron provider and fixed permissions on failed-run-data.json.
  • Justin Schumacher fixed a bug that was preventing the OS X user resource from setting the gid.
  • Matthew Kent refactored knife client create to return the correct exit code on failure.
  • Tim Potter made the ‘knife node tag delete’ output consistent with ‘knife node tag create’ and added tags to the ‘knife node show’ output.
  • Raf Geens fixed the –json-attributes flag when using the ubuntu12.04-gems bootstrap template.
  • Juanje Ojeda and Avishai Ish-Shalom made it so the deploy provider can create directories if they do not exist.
  • Mathieu Sauve-Frankel fixed knife ssh cssh to run on additional platforms.
  • Max Horbul added support for loading a data bag by symbol.
  • Deepak Kannan added an option to set the default knife verbosity in the config file.
  • Dan Crosta configured NetBSD to use the FreeBSD service provider and provided a NetBSD group provider.
  • Sriram Devadas improved the ordering of objects in the WebUI.
  • Mark Roddy fixed a bug where knife bootstrap was failing when not provided ohai hints.
  • Gardner Bickford cleaned up some unfriendly language in the unit tests.
  • Chris Parsons provided a workaround for a Net::SSH bug when passing unicode data.
  • Arthur Gautier made it so you can search for values that contain the @ symbol and fixed a bug in the webui when it tries to display special files.
  • Miguel Cabeça fixed a bug in the apt provider to allow installing packages from a different release.
  • Josiah Kiehl improved parts of knife to use its internal ui object more consistently.
  • Wes Morgan solved an issue where Chef::Knife::Ssh was being used before it was require’d.
  • Daniel Condomitti added options to knife status for filtering and reverse sorting.

Special thanks to Justin Witrick for why-run testing.

Release Notes

Bug

  • [CHEF-1247] – logrotate scripts attempt to rotate non-existent log files
  • [CHEF-1554] – knife ssh using incorrect hostname
  • [CHEF-1735] – FreeBSD service provider cannot determine rc variable name properly
  • [CHEF-1994] – The notifies syntax introduced in 0.9.10 is not implemented for subscribes
  • [CHEF-2627] – Knife SSH should return exit code based on whether or not ssh command is successful or not
  • [CHEF-2706] – The user resource does not include the -rm parameter on the usermod command when "supports :manage_home=>true"
  • [CHEF-2807] – OSX User resource provider does not properly update gid
  • [CHEF-2848] – Omnibus (embedded) fails the 'users' cookbook's need for ruby-shadow
  • [CHEF-2871] – Omnibus should be updated to use Ruby 1.9.3/Rubygems 1.8.10
  • [CHEF-2900] – chef-client documentation claims support for '–verbose'
  • [CHEF-2987] – Webui doesn't delete nodes in Firefox
  • [CHEF-3045] – Chef errors out with 403 when retrieving cookbook_file, template resources on a very long Chef run
  • [CHEF-3088] – Execute resource should accept command Arrays
  • [CHEF-3095] – mixlib-authentication fails to sign requests for very long host / user names
  • [CHEF-3132] – Knife bootstrap should require chef/knife/ssh before calling a class method on Chef::Knife::Ssh
  • [CHEF-3134] – Permissions to open for failed-run-data.json
  • [CHEF-3164] – gem_package poor interaction with omnibus via memoized Gem environment variables
  • [CHEF-3178] – cron provider does not support month names and weekday names
  • [CHEF-3183] – Consistency and expected behavior of resource notifications
  • [CHEF-3201] – knife client create – already exists exit code
  • [CHEF-3210] – wrong regexp in provider/service/freebsd.rb
  • [CHEF-3235] – [regression] file(…).owner and file(…).mode returns nil instead of expected integer value
  • [CHEF-3237] – Expanding '~/Library/LaunchAgents' fails resolving HOME when running chef-client as root
  • [CHEF-3245] – usermod group provider does not work on solaris2 platform
  • [CHEF-3261] – Use ui object instead of puts in knife bootstrap
  • [CHEF-3266] – Apt provider still can't install packages from different default release
  • [CHEF-3273] – Webui returning "undefined method …
  • [CHEF-3276] – rubygem's provider incorrectly handles options passed as a Hash if gem_binary attribute is specified.
  • [CHEF-3279] – undefined method describe_error…
  • [CHEF-3280] – Permit chef to run solr searches with @ in terms
  • [CHEF-3286] – requires_root rspec test conditional broken by shared_context
  • [CHEF-3288] – Running knife with unicode input gives rise to error: "data not multiple of block length (OpenSSL::Cipher::CipherError)"
  • [CHEF-3301] – NameError when running chef-client as a service on Windows Server 2008 R2
  • [CHEF-3302] – NoMethodError: undefined method `library_load_failed'
  • [CHEF-3309] – Error in knife bootstrap templates when no ohai hints
  • [CHEF-3311] – knife bootstrap -j doesn't work with Ubuntu 12.04 template
  • [CHEF-3320] – Chef::Provider::Service::Upstart does not set start_command
  • [CHEF-3329] – Omnibus installer fails on libpopt.so.0 rsync error
  • [CHEF-3336] – stop_clock NilClass exception on Windows
  • [CHEF-3338] – Mdadm provider fails to load current resource if the system contains an unrelated inactive array
  • [CHEF-3340] – Chef WebUI reveals environment select box list on failed user logins
  • [CHEF-3353] – tests fail on FreeBSD
  • [CHEF-3360] – pkg info may show version in parenthesis in addition to the bare string
  • [CHEF-3362] – knife bootstrap should check for templates before it builds a server
  • [CHEF-3380] – chef-client fails when attempting to disable a service that doesn't exist.
  • [CHEF-3382] – Backtrace filtering does not work correctly with chef-solo
  • [CHEF-3383] – "validate_cookbook" used knife cookbook site share instead of "validate_cookbooks"
  • [CHEF-3397] – whyrun not working with apt_repository provider
  • [CHEF-3410] – Upstart provider erroneously requires start_command, stop_command, and restart_command to be manually set
  • [CHEF-3418] – Chef won't start with Bunny 0.8.0

Improvement

  • [CHEF-630] – Deploy should create the directories it needs if they don't exist
  • [CHEF-2374] – depoy resource: number of releases kept should be configurable
  • [CHEF-2496] – OpenIndiana/OpenSolaris/Solaris 11 IPS Package Support
  • [CHEF-2731] – knife cookbook install should have an option to use current branch
  • [CHEF-2978] – Display node tags in "knife node show"
  • [CHEF-2979] – Make "knife node tag delete" output consistent with "knife node tag create"
  • [CHEF-3085] – Make knife ssh cssh platform agnostic
  • [CHEF-3092] – knife cookbook upload -a should batch uploads
  • [CHEF-3116] – Use FreeBSD service provider on NetBSD
  • [CHEF-3117] – Clients should be able to delete themselves regardless of whether they are admin
  • [CHEF-3130] – Add filtering and sorting options to knife status
  • [CHEF-3150] – specs should be shipped in gem
  • [CHEF-3186] – Separate UI from functionality in knife cookbook site download
  • [CHEF-3202] – Items are not ordered in Chef web UI list
  • [CHEF-3207] – Allow the use of a symbol when specifying a databag to load with Chef-solo
  • [CHEF-3234] – add verbosity option to knife config file
  • [CHEF-3254] – Run garbage collector before sleeping
  • [CHEF-3305] – chef-server Gemfile improvements
  • [CHEF-3321] – Initial version of cookbooks should be 0.1.0
  • [CHEF-3322] – Cookbooks should include a CHANGELOG.md
  • [CHEF-3331] – Shef DSL doesn't allow manipulation of environments
  • [CHEF-3364] – Fix up Smartos Package provider
  • [CHEF-3368] – Chef Unit Test Contains Informal Language

New Feature

  • [CHEF-13] – Add –noop support
  • [CHEF-2739] – knife exec search configurable directories
  • [CHEF-3101] – Use group {add,info,mod,del} and user {add,info,mod,del} on NetBSD for group provider
  • [CHEF-3104] – Adds fork support for chef runs
  • [CHEF-3160] – knife bootstrap should be able to create hint files and receive notification to do so from plugins
  • [CHEF-3359] – Add default providers for OmniOS platform

Archives