Below is a partial list of changes. The man page was updated as changes were made but not as much effort was put into tracking the changes, as the Git log easily does that.

chyves is a direct code descendent of iohyve and a complete code rewrite was started in April 2016. This code was forked from iohyve at version 0.7.5 “Tennessee Cherry Moonshine Edition” release at commit 2ff5b50. Nearly every line of code has changed and that reflects in the user experience. Some of the changes include: the command syntax, user properties, property storage methods, and features just to name a few changes. See the CHANGELOG or commit log for details.

General changes

• Rewrite of every function, either in part or completely. There are also 108 new functions, primarily for backend handling (user input, repetitive code, etc.).
• Better error handling.
• Kernel modules are loaded by default.
• Library files with some files only loading when needed. Index in header of sbin/chyves.
• Logs are now kept. There is a master log which is stored on the primary pool in a monthly format (YYYYMM.log). There is also guest log which are specific to that guest (work-in-progress, please report bugs). See LOGS section in man page.
• The number of parameters for each command is checked and verified for user input. When conditions are not met or user input is out of specification, a warning is displayed and chyves exits before causing damage or spewing errors messages.
• The man page has been expanded and there is a pretty online HTML version. Thank you to the ronn project for the new man page parsing platform.
• Multiple pool configurations are completely supported.
  • Pools have roles including primary, secondary, and offline. The offline role is still being developed and refined, soon to be released.
• Version all the things! Guests and dataset have versions associated with them. This is to allow upgrading from the pre-release versions all the way to version 999.99.999 possible without a hick-up, hopefully.
  • Datasets/pools get a version to ensure the dataset has the latest global and default properties, as well as structure.
  • Each guest has a version associated with it to ensure the properties are the latest. This also makes migrating a guest from another system and version possible.
  • During the upgrade process for guests or dataset, actions are performed on each to ensure compatibility. This makes the UPGRADE.md document a matter of reference.
• Self upgrades to the latest version on GitHub by running chyves upgrade.
• Properties are no longer stored as ZFS user properties, this was done with some reluctance. However the performance is much better using key values stored in hidden files and as a side benefit zfs history no longer fills up with noise. There is still a heavy dependency on ZFS due to the use of ZFS volumes as the default storage mechanism for guests’ disks and also due to the way the internal mechanisms are written.
  • Many guest properties have been added, some changed, and some have been deprecated such as name, size, and persist.
    • See:
      • chyves <guest> set <property>=<value>
      • chyves <guest> get <property>
      • chyves <guest> get all
      • chyves list properties
      • chyves list <property>
      • or the Guest Properties section in man page.
  • Default guest properties are user changeable.
    • See:
      • chyves list defaults
      • chyves defaults set <property>=<value>
      • or the Default Guest Properties section in man page.
  • There are now global properties that influence how chyves operates.
    • See:
      • chyves list global
      • chyves global set <property>=<value>
      • or the Global Configuration Properties section in man page.
• Networking changes.
  • bridge0 is no longer hard coded.
  • Complex network designs are supported.
    • Multiple tap interfaces and bridges can be assigned to a guest. Tap interfaces are assigned to guests and bridges are assigned tap interfaces and external interfaces. This information is stored as a global property.
    • Each bridge has a physical or VLAN assigned to it or can be set as a private bridge with no external interface (sensitive database traffic).
  • Two network modes: ‘auto’ and ‘system’. The ‘auto’ mode handles all configurations of the devices above the physical or VLAN interface. The ‘system’ mode relies on the user to correctly configure everything below the tap interface.
  • VALE support.
  • Emulated Intel E1000 support on 12-CURRENT hosts.
• CPU checks are ran preflight to ensure that the necessary hardware virtualization CPU features are in place such as Second Level Address Translation (SLAT), also known as nested paging.
  • Intel (Xeon 5500) CPUs are restricted to only running one core FreeBSD guests due to the lack of a feature Intel later implemented called unrestricted guest.
  • Before starting guests with PCI-passthrough devices, an IO-MMU check is ran to see if the CPU supports the feature and that the feature is enabled.
  • See DEPENDENCIES section in man page for details.
• make directives
  • make clean - Removes chyves.8.gz file for developers using git
  • make docs - Runs the ronn command to build the roff and html document files. Requires Ruby gem ronn.
  • make install - Installs chyves to ‘/usr/local’.
  • make installrc - Enables ‘chyves_enable=YES’ in ‘/etc/rc.conf’ using ‘sysrc’.
  • make deinstall - Remove chyves from ‘/usr/local’.
  • make rcremove - Removes ‘chyves_enable=YES’ from ‘/etc/rc.conf’ using ‘sysrc’.
  • make what - Prints available help directives.

Guest changes

• UEFI guests have been heavily tested and are treated as a first class citizen.
  • Have more functionality compared to the other loader types.
    • UEFI GOP support.
  • Part of this is due to the complete rewrite of the start command.
• Raw file backed disks are supported. All files in the /chyves/<pool>/guests/<guest>/img folder or child folders are attached to the guest.
• Clone functionality has been expanded.
  • True ZFS clones are now supported and only the disks volumes are cloned. Independent clones are also possible and copied data using zfs send | zfs recv.
  • Clones can optionally have new unique properties assigned to them while cloning where the (complex) network design is duplicated, a new UUID is assigned, and a new serial console interface number is also assigned. Or clones can keep the exact properties as an offline backup. Clones with the exact properties can be deleted with chyves <guest> delete keepnet to keep the network associations.
• There is now a feature called “Multi-guest’ which allows for rapid execution of a command over several guests machines in one command.
  • Command compatible with multi-guest are indicated in the SYNOPSIS section in the man page with “MG”.
    • For example chyves gst1,gst2,gst3 set cpu=8 will set the cpu property to “8” for guests “gst1”, “gst2”, and “gst3”.
  • Some commands support the keyword “all” to specify all guests on system.
    • For example chyves all stop or chyves all reclaim.
  • This functionality will be expanded in the future to include the use of group memberships.
• Deprecated following user properties
  • persist
  • con -> serial
  • name
  • size

All of this does have a cost and the cost is slowness when executing commands.

Command syntax quick reference and comparison:

The man page (html version) has been vastly expanded and is a recommended read. Below is a table containing the old iohyve command and syntax compared to the new chyves commands and syntax (at the time of release.):