Charm Tools documentation¶
The charm command includes several subcommands used to build, maintain, and release Juju Charms, which are Open Source encapsulated operations logic for managing software in the cloud or bare-metal servers using cloud-like APIs.
Installation is easy with snaps:
snap install --classic charm
Reference for the various available commands can be found below, or via the command-line with:
charm help
Available Commands¶
The following subcommands are available and can be invoked as charm <command>
(for example, charm build
). Details for each command, including the supported
options and parameters, can be output with either charm help <command>
or
charm <command> --help
.
Command | Description |
---|---|
add |
add icon, readme, or tests to a charm |
attach |
upload a file as a resource for a charm |
attach-plan |
associates the charm with the plan |
build |
build a charm from layers and interfaces |
create |
create a new charm |
grant |
grant charm or bundle permissions |
help |
Show help on a command or other topic. |
layers |
Show a colored breakdown of what layers each file came from |
list |
list charms for the given users. |
list-plans |
list plans |
list-resources |
display the resources for a charm in the charm store |
login |
login to the charm store |
logout |
logout from the charm store |
proof |
perform static analysis on a charm or bundle |
pull |
download a charm or bundle from the charm store |
pull-resource |
pull a charm resource to the local machine |
push |
push a charm or bundle into the charm store |
push-plan |
push new plan |
push-term |
create new Terms and Conditions document (revision) |
release |
release a charm or bundle |
release-term |
releases the given terms document |
resume-plan |
resumes plan for specified charms |
revoke |
revoke charm or bundle permissions |
set |
set charm or bundle extra-info, home page or bugs URL |
show |
print information on a charm or bundle |
show-plan |
show plan details |
show-plan-revisions |
show all revision of a plan |
show-term |
shows the specified term |
suspend-plan |
suspends plan for specified charms |
terms |
list terms owned by the current user |
terms-used |
list terms required by current user’s charms |
version |
display tooling version information |
whoami |
display jaas user id and group membership |
Build Tactics¶
When building charms, multiple layers are brought together in an ordered,
depth-first recursive fashion. The individual files of each layer are merged
according to a list of merge tactics. These tactics determine whether the file
from a higher layer will replace or be merged with the copy from the lower
layer, with the details of how the merge happens being implemented by the
tactic. Each file is tested against each tactic in a specific order (as
determined by the DEFAULT_TACTICS
list), with the first one to match being
applied to the file and all other tactics disregarded.
Built-in Tactics¶
ActionsYAML |
Tactic for processing and combining the actions.yaml file from each layer. |
ConfigYAML |
Tactic for processing and combining the config.yaml file from each layer. |
CopyTactic |
Tactic to copy a file without modification or merging. |
CopyrightTactic |
Tactic to combine the copyright info from all layers into a final machine-readable format. |
DistYAML |
Tactic for processing and combining the dist.yaml file from each layer. |
DynamicHookBind |
Base class for process hooks dynamically generated from the hook template. |
ExactMatch |
Mixin to match a file with an exact name. |
ExcludeTactic |
Tactic to handle per-layer excludes. |
IgnoreTactic |
Tactic to handle per-layer ignores. |
InstallerTactic |
Tactic to process any .pypi files and install Python packages directly into the charm’s lib/ directory. |
InterfaceBind |
Tactic to copy the hook template into place for all relation hooks. |
InterfaceCopy |
Tactic to process a relation endpoint using an interface layer. |
JSONTactic |
Base class for tactics dealing with JSON data. |
LayerYAML |
Tactic for processing and combining the layer.yaml file from each layer. |
ManifestTactic |
Tactic to avoid copying a build manifest file from a base layer. |
MetadataYAML |
Tactic for processing and combining the metadata.yaml file from each layer. |
ResourcesYAML |
Tactic for processing and combining the resources.yaml file from each layer. |
SerializedTactic |
Base class for tactics which deal with serialized data, such as YAML or JSON. |
StandardHooksBind |
Tactic to copy the hook template into place for all standard hooks. |
StorageBind |
Tactic to copy the hook template into place for all storage hooks. |
Tactic |
Base class for all tactics. |
VersionTactic |
Tactic to generate the version file with VCS revision info to be displayed in juju status . |
WheelhouseTactic |
Tactic to process the wheelhouse.txt file and build a source-only wheelhouse of Python packages in the charm’s wheelhouse/ directory. |
YAMLTactic |
Base class for tactics dealing with YAML data. |
extend_with_default |
Extend a jsonschema validator to propagate default values prior to validating. |
load_tactic |
Load a tactic from the current layer using a dotted path. |
-
class
charmtools.build.tactics.
ActionsYAML
(*args, **kwargs)¶ Tactic for processing and combining the
actions.yaml
file from each layer.
-
class
charmtools.build.tactics.
ConfigYAML
(*args, **kwargs)¶ Tactic for processing and combining the
config.yaml
file from each layer.
-
class
charmtools.build.tactics.
CopyTactic
(entity, target, layer, next_config)¶ Tactic to copy a file without modification or merging.
The last version of the file “wins” (e.g., from the charm layer).
This is the final fallback tactic if nothing else matches.
-
class
charmtools.build.tactics.
CopyrightTactic
(*args, **kwargs)¶ Tactic to combine the copyright info from all layers into a final machine-readable format.
-
class
charmtools.build.tactics.
DistYAML
(*args, **kwargs)¶ Tactic for processing and combining the
dist.yaml
file from each layer.
-
class
charmtools.build.tactics.
DynamicHookBind
(name, owner, target, config, output_files, template_file)¶ Base class for process hooks dynamically generated from the hook template.
This tactic is not used directly, but serves as a base for the type-specific dynamic hook tactics, like
StandardHooksBind
, orInterfaceBind
.-
HOOKS
= []¶ List of all hooks to populate.
-
sign
()¶ Sign all hook files generated by this tactic.
-
-
class
charmtools.build.tactics.
ExactMatch
¶ Mixin to match a file with an exact name.
-
FILENAME
= None¶ The filename to be matched
-
classmethod
trigger
(entity, target, layer, next_config)¶ Match if the current entity’s filename is what we’re looking for.
-
-
class
charmtools.build.tactics.
ExcludeTactic
(entity, target, layer, next_config)¶ Tactic to handle per-layer excludes.
If a given layer’s
layer.yaml
has anexclude
list, then any file or directory included in that list that is provided by the current layer will be ignored, though any matching file or directory provided by base layers or any higher level layers will be included.The
exclude
list uses the same format as a.gitignore
file.
-
class
charmtools.build.tactics.
IgnoreTactic
(entity, target, layer, next_config)¶ Tactic to handle per-layer ignores.
If a given layer’s
layer.yaml
has anignore
list, then any file or directory included in that list that is provided by base layers will be ignored, though any matching file or directory provided by the current or any higher level layers will be included.The
ignore
list uses the same format as a.gitignore
file.
-
class
charmtools.build.tactics.
InstallerTactic
(entity, target, layer, next_config)¶ Tactic to process any
.pypi
files and install Python packages directly into the charm’slib/
directory.This is used in Kubernetes type charms due to the lack of a proper install or bootstrap phase.
-
class
charmtools.build.tactics.
InterfaceBind
(name, owner, target, config, output_files, template_file)¶ Tactic to copy the hook template into place for all relation hooks.
This tactic is not part of the normal set of tactics that are matched against files. Instead, it is manually called to fill in the set of relation hooks needed by this charm.
-
class
charmtools.build.tactics.
InterfaceCopy
(interface, relation_name, role, target, config)¶ Tactic to process a relation endpoint using an interface layer.
This tactic is not part of the normal set of tactics that are matched against files. Instead, it is manually called for each relation endpoint that has a corresponding interface layer.
-
class
charmtools.build.tactics.
JSONTactic
(*args, **kwargs)¶ Base class for tactics dealing with JSON data.
-
dump
(data)¶ Serialize and write the data to the file.
Must be impelemented by a subclass.
-
load
(fn)¶ Load and deserialize the data from the file.
Must be impelemented by a subclass.
-
-
class
charmtools.build.tactics.
LayerYAML
(*args, **kwargs)¶ Tactic for processing and combining the
layer.yaml
file from each layer.The input
layer.yaml
files can contain the following sections:includes
This is the heart of layering. Layers and interface layers referenced in this list value are pulled in during charm build and combined with each other to produce the final layer.defines
This object can contain a jsonschema used to define and validate options passed to this layer from another layer. The options and schema will be namespaced by the current layer name.options
This object can contain option name/value sections for other layers.config
,metadata
,dist
, orresources
These objects can contain adeletes
object to list keys that should be deleted from the resulting<section>.yaml
.
Example, layer
foo
might define thislayer.yaml
file:includes: - layer:basic - interface:foo defines: foo-opt: type: string default: 'foo-default' options: basic: use_venv: true
And layer
bar
might define thislayer.yaml
file:includes: - layer:foo options: foo-opt: 'bar-value' metadata: deletes: - 'requires.foo-relation'
-
class
charmtools.build.tactics.
ManifestTactic
(entity, target, layer, next_config)¶ Tactic to avoid copying a build manifest file from a base layer.
-
class
charmtools.build.tactics.
MetadataYAML
(*args, **kwargs)¶ Tactic for processing and combining the
metadata.yaml
file from each layer.
-
class
charmtools.build.tactics.
ResourcesYAML
(*args, **kwargs)¶ Tactic for processing and combining the
resources.yaml
file from each layer.
-
class
charmtools.build.tactics.
SerializedTactic
(*args, **kwargs)¶ Base class for tactics which deal with serialized data, such as YAML or JSON.
-
apply_edits
()¶ Apply any edits defined in the final
layer.yaml
file to the data.An example edit definition:
metadata: deletes: - requires.http
-
combine
(existing)¶ Merge the deserialized data from two layers using
deepmerge
.
-
dump
(data)¶ Serialize and write the data to the file.
Must be impelemented by a subclass.
-
load
(fn)¶ Load and deserialize the data from the file.
Must be impelemented by a subclass.
-
process
()¶ Now that the tactics for the current entity have been combined for all layers, process the entity to produce the final output file.
Must be implemented by a subclass.
-
read
()¶ Read and cache the data into memory, using
self.load()
.
-
-
class
charmtools.build.tactics.
StandardHooksBind
(name, owner, target, config, output_files, template_file)¶ Tactic to copy the hook template into place for all standard hooks.
This tactic is not part of the normal set of tactics that are matched against files. Instead, it is manually called to fill in the standard set of hook implementations.
-
class
charmtools.build.tactics.
StorageBind
(name, owner, target, config, output_files, template_file)¶ Tactic to copy the hook template into place for all storage hooks.
This tactic is not part of the normal set of tactics that are matched against files. Instead, it is manually called to fill in the set of storage hooks needed by this charm.
-
class
charmtools.build.tactics.
Tactic
(entity, target, layer, next_config)¶ Base class for all tactics.
Subclasses must implement at least
trigger
andprocess
, and probably also want to implementcombine
.-
combine
(existing)¶ Produce a tactic informed by the existing tactic for an entry.
This is when a rule in a higher level charm overrode something in one of its bases for example.
Should be implemented by a subclass if any sort of merging behavior is desired.
-
config
¶ Return the combined config from the layer above this (if any), this, and all lower layers.
Note that it includes one layer higher so that the tactic can make decisions based on the upcoming layer.
-
current
¶ Alias for
Tactic.layer
-
entity
¶ The current entity (a.k.a. file) being processed.
-
classmethod
get
(entity, target, layer, next_config, current_config, existing_tactic)¶ Factory method to get an instance of the correct Tactic to handle the given entity.
-
layer
¶ The current layer under consideration
-
layer_name
¶ Name of the current layer being processed.
-
lint
()¶ Test the resulting file to ensure that it is valid.
Return
True
if valid. If invalid, returnFalse
or raise aBuildError
Should be implemented by a subclass.
-
process
()¶ Now that the tactics for the current entity have been combined for all layers, process the entity to produce the final output file.
Must be implemented by a subclass.
-
read
()¶ Read the contents of the file to be processed.
Can be implemented by a subclass. By default, returns
None
.
-
relpath
¶ The path to the file relative to the layer.
-
sign
()¶ Return signature in the form
{relpath: (origin layer, SHA256)}
Can be overridden by a subclass, but the default implementation will usually be fine.
-
target
¶ The target (final) layer.
-
target_file
¶ The location where the processed file will be written to.
-
classmethod
trigger
(entity, target, layer, next_config)¶ Determine whether the rule should apply to a given entity (file).
Generally, this should check the entity name, but could conceivably also inspect the contents of the file.
Must be implemented by a subclass or the tactic will never match.
-
-
class
charmtools.build.tactics.
VersionTactic
(charm, target, layer, next_config)¶ Tactic to generate the
version
file with VCS revision info to be displayed injuju status
.This tactic is not part of the normal set of tactics that are matched against files. Instead, it is manually called to generate the
version
file.
-
class
charmtools.build.tactics.
WheelhouseTactic
(*args, **kwargs)¶ Tactic to process the
wheelhouse.txt
file and build a source-only wheelhouse of Python packages in the charm’swheelhouse/
directory.-
read
()¶ Read the contents of the file to be processed.
Can be implemented by a subclass. By default, returns
None
.
-
-
class
charmtools.build.tactics.
YAMLTactic
(*args, **kwargs)¶ Base class for tactics dealing with YAML data.
Tries to ensure that the order of keys is preserved.
-
dump
(data)¶ Serialize and write the data to the file.
Must be impelemented by a subclass.
-
load
(fn)¶ Load and deserialize the data from the file.
Must be impelemented by a subclass.
-
-
charmtools.build.tactics.
extend_with_default
(validator_class)¶ Extend a jsonschema validator to propagate default values prior to validating.
Used internally to ensure validation of layer options supports default values.
Custom Tactics¶
A charm or layer can also define one or more custom tactics in its layer.yaml
file. The file can contain a top-level tactics
key, whose value is a list of
dotted Python module names, relative to the layer’s base directory. For
example, a layer could include this in its layer.yaml
:
tactics:
- tactics.my_layer.READMETactic
This would cause the build command to look for a module tactics/my_layer.py
with a class of READMETactic
in it, which must inherit from
Tactic
.
Custom tactics are tested before the built-in tactics, so they can override the behavior of built-in tactics if desired. Care should be taken if doing this because changing the behavior of built-in tactics can end up breaking other layers or charms.
Reproducible Charms¶
When building charms, multiple layers are brought together in an ordered,
depth-first recursive fashion. The individual files of each layer are merged,
and then python modules are brought in according to wheelhouse.txt
files
that may exist in each layer.
Layers (and Interfaces) are typically Git repositories, and by default the
default branch (usually called master
) of the repository is fetched and
used.
Also, although the top level Python modules can be pinned in the
wheelhouse.txt
files, any dependent modules are fetched as their latest
versions. This makes re-building a charm with the same layers and modules
tricky, which may be required for stable charms. It is possible, by populating
layer and interface directories directly, and by pinning every Python module in
a wheelhouse.txt
override file that is passed using the
--wheelhouse-overrides
option to the charm build
command.
An alternative strategy is to use a new feature of the charm build
command
which can generate a lock file that contains all of the layers and Python
modules, and their versions. This can then, for subsequent builds, be used to
fetch the same layer versions and Python modules to re-create the charm.
As the lock file is a JSON
file, it can be manually edited to change a
layer version if a new version of a stable charm is needed, or a python module
can be changed.
Additionally, it is possible to track a branch in the repository for a layer so that a stable (or feature) branch can be maintained and then charms rebuilt from that branch.
The new options for this feature are:
--write-lock-file
--use-lock-file-branches
--ignore-lock-file
Creating the lock file¶
To create a lock file, the option --write-lock-file
is passed to the
charm build
command. This option automatically ignores the existing lock
file, and rebuilds the charm using the latest versions of the layers and the
versions of the modules as determined in the various wheelhouse.txt
files.
Python module versions are also recorded. If a VCS repository is used for the python module, then any branch specified is also recorded, along with the commit.
At the end of the build, the lock file is written with all of the layer and Python module information.
The lock file is installed in the base layer directory so that it can be committed into the VCS and used for subsequent builds.
The name of the lock file is build.lock
.
Rebuilding the charm from the lock file¶
If a lock file (build.lock
) is available in the top layer, then it will be
used to control the versions of the layers and modules by default. i.e. the
presence of the lock file controls the build.
Three options are available which can influence the build when a lock file is present:
--ignore-lock-file
--use-lock-file-branches
--wheelhouse-overrides
If the --ignore-lock-file
option is used, then the charm is built as though
there is no lock file.
If the --use-lock-file-branches
is used, then, for VCS items (layers,
interfaces, and Python modules specified using a VCS string), then the branch
(if there was one) will be used, rather than the commit version. This can be
used to track a branch in a layer or Python module.
Note: if --wheelhouse-overrides
is used, then that wheelhouse will override
the lock file. i.e. the lock file overrides the layers’ wheelhouse.txt
file, and then the --wheelhouse-overrides
then can override the lock-file.
This is intentional to allow the build to perform specific overrides as
needed.
Other useful information¶
This is the first iteration of ‘reproducible charms’. As such, only Git is supported as the VCS for the layers, and Git and Bazaar for Python modules. A future iteration may support more VCS systems.
Only the top layer is inspected for a build.lock
file. Any other layers
are considered inputs and their build.lock
files are ignored (if they are
present).
Also, regardless of the wheelhouse.txt
layers, the lock file will override
any changes that may be introduced in stable branches, if they are bing tracked
using --use-lock-file-branches
. This may lead to unexpected behaviour.
Contributing¶
The charm command is created from the combination of two repositorires:
- charm-tools handles local operations, such as building and linting charms
- charmstore-client handles interactions with the charm store, such as pushing or pulling charms, or managing access controls
Bugs should be filed against the appropriate repository for the most efficient handling.
Changelog¶
charm-tools 2.8.2 + charmstore-client 2.5.0¶
Monday February 1 2021
charm-tools
- Fix reproducible charms issues (#598)
charm-tools 2.8.1 + charmstore-client 2.5.0¶
Wednesday January 27 2021
charm-tools
- Add option to create .charm file (#592)
- Add ‘docs’ to known metadata fields (#591)
- Add reproducible charm build feature (#585)
- Fix exception rendering “already promulgated” error (#590)
- Align setup.py to requirements.txt (#589)
- Fix TypeError from linter on X.Y min-juju-version (#588)
- Make output_dir the same as build_dir (#564)
charm-tools 2.8.0 + charmstore-client 2.5.0¶
Tuesday November 10 2020
charm-tools
- Fix snap build for updated charmstore-client (#587)
- Store rev when pull-source on a subdir layer (#583)
- Add revision info to output of pull-source (#582)
- Add –branch option to pull-source (#581)
- Raise more useful BuildError on missing pkg name (#579)
- Deprecate Operator charm template (#578)
charmstore-client
- Update dependencies
- Make charm-push support archives
charm-tools 2.7.8 + charmstore-client 2.4.0+git-13-547c6f2¶
Tuesday July 21 2020
charm-tools
- Normalize package names when processing wheelhouse (#576)
charm-tools 2.7.7 + charmstore-client 2.4.0+git-13-547c6f2¶
Monday July 20 2020
charm-tools
- Fix handling of comments in wheelhouse (#574)
charm-tools 2.7.6 + charmstore-client 2.4.0+git-13-547c6f2¶
Thursday July 16 2020
charm-tools
- Switch to requirements-parser for wheelhouse (#572)
charm-tools 2.7.5 + charmstore-client 2.4.0+git-13-547c6f2¶
Thursday June 25 2020
charm-tools
- Process wheelhouse.txt holistically rather than per-layer (#569)
- Handle invalid config file more gracefully (#567)
- Default to charming category of the Juju Discourse (#565)
charm-tools 2.7.5 + charmstore-client 2.4.0+git-13-547c6f2¶
Thursday June 25 2020
charm-tools
- Process wheelhouse.txt holistically rather than per-layer (#569)
- Handle invalid config file more gracefully (#567)
- Default to charming category of the Juju Discourse (#565)
charm-tools 2.7.4 + charmstore-client 2.4.0+git-13-547c6f2¶
Thursday March 26 2020
charm-tools
- Add workaround for user site package conflicts (#561)
- Add Build Snap action so PRs have snap to test easily (#562)
charm-tools 2.7.3 + charmstore-client 2.4.0+git-13-547c6f2¶
Saturday Feb 29 2020
charm-tools
- Add Operator charm template (#557)
- Add OpenStack templates to requirements (#558)
- Fix 471 (#556)
- Add functions support; (#555)
- Allow boolean config options to have null default (#554)
charmstore-client
- fix dependencies
- cmd/charm: allow users with domains in ACLs
- Updated charmstore and charmrepo dependency.
- charm whoami: return an error when the user is not logged in
- Update dependencies
- Fix dependency files
charm-tools 2.7.2 + charmstore-client 2.4.0+git-3-cbbf887¶
Tuesday October 8 2019
charm-tools
- Add opendev.org https and git fetcher (#553)
charmstore-client
- Disallow release in promulgated namespace
charm-tools 2.7.1 + charmstore-client 2.4.0¶
Tuesday September 24 2019
charm-tools
- Fix maintainer validation not handling unicode (#550)
- Fix snap builds on other arches (#548)
- Change deployment.type optional (for k8s charms) (#547)
- Move daemonset to deployment.type (for k8s charms) (#546)
charm-tools 2.7.0 + charmstore-client 2.4.0¶
Wednesday September 18 2019
charm-tools
- Fix charm-build conflict when building concurrently (#545)
- Rename README files with markdown extension (#543)
- Update charm.1 manpage (#522)
- Feature/add deployment field2metadata (#544)
- fix charm build help message (#542)
- Cleanup cached layers / interfaces after build (#540)
- edge case for setting charm_ver (#538)
charm-tools 2.6.1 + charmstore-client 2.4.0¶
Thursday July 11 2019
charm-tools
- Remove bad URL from PR template (#537)
- Update pypi release target to work with newer tox (#530)
- requirements.txt: update version limit for requests (#535) (#536)
- Fix config key regexp to allow short config keys. (#534)
charm-tools 2.6.0 + charmstore-client 2.4.0¶
Thursday June 6 2019
charm-tools
- Honor ignores / excludes when checking for post-build changes (#529)
- Resolve vergit runtime dependency (#527)
- Upgrade to use py3.7 on Travis (#523)
- Fix installing from git without vergit installed (#520)
- Fix installation dependency on vergit (#519)
- Gracefully handle JSON decode errors from layer index (#516)
- Add support for layer-index and fallback-layer-index (#515)
- Ensure setuptools for charmstore-client build (#509)
- Refactor version handling in snap to work with core18 (#508)
- Make series required (#499)
- Add setuptools to requirements.txt (#498)
- Fix charm-layer handling of old format build-manifest (#496)
- Fix nested build dir check in Python2 (#494)
- Improve docs for LayerYAML tactic (#493)
- Add promulgate and unpromulgate commands (#491)
- Fix and improve charm-layers (#492)
- Fix checking of build dir nested under source dir (#490)
- Add basic documentation (#489)
- Allow build folders in the charm (#486)
- Fix CHARM_HIDE_METRICS environment variable (#483)
- Address security alerts from GitHub (#484)
- Use shutil.copytree instead of path.rename (#482)
charmstore-client
- Remove the temporary file
- update charmrepo dependency
- update charm dependency
- internal/ingest: set permissions correctly
- cmd/charm-ingest: use –hardlimit not –softlimit
- cmd/charm-ingest: expose disk limits
- make tests pass
- internal/ingest: transfer resources
- cmd/charm-ingest: Add a basic ingest command
- internal/ingest: resolve resources in whitelist
- internal/ingest: expose public ingest API.
- cmd/charm-ingest: Add the basics of whitelist parsing
- restore go-cmp dependency version
- Move cmd/ingest to internal/ingest
- cmd/ingest: fix comment from previous review
- cmd/ingest: run tests against real charmstore servers
- cmd/ingest: core ingestion logic
- cmd/charm/charmcmd: add some basic tests for show command
- cmd/charm/charmcmd: improve output in charm show for unpublished charms
- cmd/ingest: new ingest command
- cmd/charm/charmcmd: improve incompatible registry version error
- Update usage of docker to oci-image resource type.
- Reviews.
- cmd/charmcmd: Better yaml output for resources.
- cmd/charmcmd: Allow multiple users in list.
- all: use quicktest for tests