pyinfra v0.8

Another major internal release that re-works how operations are ordered such that operation arguments no longer need to be the same for each host. This works by using the line numbers on which operations are defined, nesting as required.

• Getting started
• Documentation

pyinfra v0.7

This is a huge step forward for pyinfra; as well as a bunch of key new modules and a new progress bar that highlights what's pending, pyinfra now compiles deploy code so if statements work as expected (no need to use the with state.limit workaround).

• Getting started
• Documentation

Changes in 0.7:

• Add mysql module
  • Operations: mysql.sql, mysql.user, mysql.database, mysql.privileges, mysql.dump, mysql.load
  • Facts: mysql_databases, mysql_users, mysql_user_grants
• Add postgresql module
  • Operations: postgresql.sql, postgresql.role, postgresql.database, postgresql.dump, postgresql.load
  • Facts: postgresql_databases, postgresql_roles
• Add puppet module with puppet.agent operation (@tobald)
• Add server.crontab, server.modprobe and server.hostname operations
• Add git.config operation
• Add kernel_modules, crontab and git_config facts
• Add global install virtualenv support (like iPython)
• Massively improved progress bar which highlights remaining hosts and tracks progress per operation or fact
• Improved SSH config parsing, including proxyjump support (@tobald)
• Support for CONFIG variables defined in local.include files
• Fix command fact now outputs everything not just the first line

Internal changes:

• Replace --debug-state with --debug-operations and --debug-facts
• pyinfra now compiles the top-level scope of deploy code, meaning if statements no longer generate imbalanced operations
  • This means the recommendations to use state.when in place of conditional statements is invalid
  • Updated the warning shown, now once, with a link
  • Included a test deploy_branches.py which can be used to verify operations do run in order for each host when compile is disabled
  • Compiling can be disabled by setting PYINFRA_COMPILE=off environment variable
• Deprecate state.limit and replace with state.hosts(hosts) (consistency with global operation kwarg hosts not limit)
• Major internal refactor of AttrData handling to reduce operation branching:
  • Generate AttrData on creation, rather than read
  • Add nesting support for AttrData so host.data.thing['X'] will not create branching operations
  • Turn fact data into AttrData
  • Make host.name an AttrDataStr
  • Hash True, False and None constants as the same so they can change between hosts without branching operations
  • Update docs and warning on operation branching
• Better default for pool parallel size
• Show stdout if stderr is empty on command failure (surprisingly common)

pyinfra v0.6.1

• Getting started
• Documentation

Changes in 0.6.1:

• Fix file "uploading" for the @local connector

pyinfra v0.6

• Getting started
• Documentation

Changes in 0.6:

• Make --limit apply the limit similarly to state.limit
  • makes it possible to execute facts on hosts outside the --limit
  • --limit no longer alters the inventory, instead provides an "initial" state limit
• Add when=True kwarg to local.include
• Make it possible to add data to individual hosts in @vagrant.json configuration files
• Add memory and cpus facts
• Refactor how we track host state throughout deploy
• Refactor facts to only gather missing ones (enabling partial gathering)
• Improve check for valid /etc/init.d/ services by looking for LSB header
• Fix boolean constant detection with AST in Python3
• Fix parsing ls output where setgid is set
• Fix sudo/su file uploads with the @local connector

pyinfra v0.5.3

• Getting started
• Documentation

Changes in 0.5.3:

• Fix writing unicode data with @local
• Capture IOErrors when SFTPing, note where remote disks might be full
• Properly serialise Host objects for --debug-state

pyinfra v0.5.2

• Getting started
• Documentation

Changes in 0.5.2:

• Add exclude_dir and add_deploy_dir kwargs to files.sync
• Add pipfile for dev
• Fix files.put when using @local

pyinfra v0.5.1

• Getting started
• Documentation

Changes in 0.5.1:

• Make environment variables stick between multiple commands
• Fix npm packages fact missing a return(!)

pyinfra v0.5

• Getting started
• Documentation

Changes in 0.5:

Vagrant connector:

# Run a deploy on all Vagrant machines (vagrant status list)
pyinfra @vagrant deploy.py
pyinfra @vagrant/vm_name deploy.py

# Can be used in tandem with other inventory:
pyinfra @vagrant,my-host.net deploy.py
pyinfra @vagrant,@local,my-host.net fact os

Consistent conditional branching:

• Add global when kwarg to all operations, similar to hosts can be used to prevent operations executing on hosts based on a condition
• Add state.limit(hosts) and state.when(condition) context managers to use in place of if statements within deploys
• @deploys and the context managers (state.limit, state.when) can all be nested as much as needed (although if you need to nest a lot, you're probably doing it wrong!)
• Add data_defaults kwarg to @deploy functions, meaning third party pyinfra packages can provide sensible defaults that the user can override individually
• Display a large warning when imbalanced branches are detected, linking the user to the documentation for the above

Examples:

from pyinfra import host, state
from pyinfra.modules import server

# Replace if blocks with the state.when context
with state.when(host.name == 'my-host.net'):
    server.shell('echo "my-host.net op!"')
    ...

# Use the when kwarg to achieve the same, for single operations
server.shell(
    'echo "my-host.net op!"',
    when=host.name == 'my-host.net',
)

with state.limit(inventory.get_host('my-host.net')):
    server.shell('echo "my-host.net op!"')
    ...

server.shell(
    'echo "my-host.net op!"',
    limit=inventory.get_host('my-host.net'),
)

Note that if statements/etc still work as before but pyinfra will print out a warning explaining the implications and linking to the docs (http://pyinfra.readthedocs.io/page/using_python.html#conditional-branches).

• Hooks change: no longer loaded from deploy files, only from config.py, due to changes from 0.4 (removal of FakeState nonsense)
• Add gpgkey argument to the yum.repo operation
• Add lsb_release fact
• apt_sources fact now supports apt repos with options ([arch=amd64])
• Improved error output when connecting
• Update testing box from Ubuntu 15 to Ubuntu 16
• Ensure ~/.ssh exists keyscanning in ssh.keyscan
• Don't include tests during setup!
• Fix caching of local SHA1s on files

pyinfra v0.4.1

• Getting started
• Documentation

Changes in 0.4.1:

• Add vzctl.unmount operation
• Add script to generate empty test files
• Increase module test coverage significantly
• Fix incorrect args in vzctl.restart operation
• Fix save=False kwarg on vzctl.set not affecting command output (always saved)
• Fix gem.packages install command

pyinfra v0.4

• Getting started
• Documentation

Changes in 0.4:

• Major change: entirely new, streamlined CLI. Legacy support will remain for the next few releases. Usage is now:

# Run one or more deploys against the inventory
pyinfra INVENTORY deploy_web.py [deploy_db.py]...

# Run a single operation against the inventory
pyinfra INVENTORY server.user pyinfra,home=/home/pyinfra

# Execute an arbitrary command on the inventory
pyinfra INVENTORY exec -- echo "hello world"

# Run one or more facts on the inventory
pyinfra INVENTORY fact linux_distribution [users]...

• Major addition: new connectors module that means hosts are no longer limited to SSH targets. Hostnames prefixed in @ define which non-SSH connector to use. There is a new local connector for executing directly on the local machine, use hostname @local, eg:

pyinfra @local fact arch

• Major addition: add @deploy wrapper for pyinfra related modules (eg pyinfra-openstack) to wrap a deploy (collection of operations) under one function, eg:

from pyinfra.api import deploy

@deploy('Install Openstack controller')
def install_openstack_controller(state, host):
    apt.packages(
        state, host,
        {'Install openstack-client'},
        ['openstack-client'],
    )
    ...

• Add SSH module to execute SSH from others hosts: ssh.keyscan, ssh.command, ssh.upload, ssh.download
• Add vzctl module to manage OpenVZ containers: vzctl.create, vzctl.stop, vzctl.start, vzctl.restart, vzctl.delete, vzctl.set
• Add on_success and on_error callbacks to all operations (args = (state, host, op_hash))
• Add server.script_template operation
• Add global hosts kwarg to all operations, working like local.include's
• Add cache_time kwarg to apt.update operation
• Add Inventory.get_group and Inventory.get_host
• Inventory __len__ now (correctly) looks at active hosts, rather than all
• Add Inventory.len_all_hosts to replace above bug/qwirk
• Add progress spinner and % indicator to CLI
• Replace docopt/termcolor with click
• Moved pyinfra.cli to pyinfra_cli (internal breaking)
• Switch to setuptools entry_points instead of distutils scripts
• Expand Travis.ci testing to Python 3.6 and 3.7 nightly
• Remove unused kwargs (sudo, sudo_user, su_user) from pyinfra.api.facts.get_facts