Contributors1 hourHabitat Builder makes building cloud-native apps faster and easier than ever.
In the Building Applications with Habitat track, you created plans that define how your software should be built. You then ran those plans locally on your system through Habitat Studio.
In this module, you'll discover how to connect your application to Builder so that we can rebuild your package for you automatically when you push a change up to GitHub. You'll also see how Builder can publish images to Docker Hub and how Builder can automatically rebuild your app when one your app's dependencies is rebuilt.
Prerequisites
Run through the Habitat Builder demo
This module builds on the Habitat demo on habitat.sh.
We recommend you run through the demo before starting this module. Doing so ensures you have:
- GitHub, Habitat Builder, and Docker Hub accounts.
- A Habitat origin.
- The Habitat Builder GitHub app.
- Docker Hub integrated with your Habitat Builder account.
- Docker on your local system.
Start the demo
Get familiar with the Habitat CLI
In this module, you use the hab command, which is the command-line interface (CLI) for Habitat.
Although not required, we recommend you work through the Try Habitat module before you continue with this module. You'll learn how to build and run packages, what the Studio is, and the role of the Supervisor.
In this module, you build a web application from starter code that we provide. You also add functionality to your app by building a utility library.
The functionality you build is similar to what you find in the Build a web application with Habitat and Build a Habitat package from source modules. Although not required, you can work through these modules to familiarize yourself with the sample app and how it works.
During these modules, you'll also learn about scaffolding and how to build a utility program from its sources.
Install Git
In this module, you use Git commands to commit and upload your changes. Install Git if you don't already have it.
Export the HAB_ORIGIN environment variable
Your origin name appears in the Habitat packages that you build. To make it easy to work with your package names, export the HAB_ORIGIN environment variable. Here's how to do that on Linux, macOS, and Windows.
Linux and macOS
Replace learn-chef with your origin name.
Terminal: ~
$ | export HAB_ORIGIN=learn-chef
|
Terminal: ~
$ | echo $HAB_ORIGINlearn-chef
|
Windows
Replace learn-chef with your origin name.
Windows PowerShell: ~
PS > | $env:HAB_ORIGIN = "learn-chef"
|
Windows PowerShell: ~
PS > | $env:HAB_ORIGINlearn-chef
|
Prepare your accounts
Because this module uses your GitHub, Habitat, and Docker Hub accounts, we recommend you open a new browser window and create a tab for each of these sites:
You'll need a Docker Hub repo so Builder can export your package as a Docker container. From Docker Hub, create a repo now. We recommend you name it habitatize.
1. Get the web app
In this part, you get a basic web application and verify that it builds and runs on your system.
In other modules, you clone an existing GitHub repo to get starter code. Here, you fork a GitHub repo because you later connect it to Builder so that we can build it for you automatically when you push changes. A fork gives you your own space to make changes without affecting the original upstream repo.
Get the source code from GitHub
Start by forking https://github.com/learn-chef/hab-web-app-builder to your GitHub account.
From your terminal, move to a working directory, for example, your home directory.
Next, clone the hab-web-app-builder. Replace GITHUB_ORG with your org name.
Terminal: ~
$ | git clone https://github.com/GITHUB_ORG/hab-web-app-builder.git
|
Move to the hab-web-app-builder directory.
Build and run the app
Next you'll enter the Studio and build the app. Before you do that, export this environment variable.
Linux and macOS
Terminal: hab-web-app-builder
$ | export HAB_DOCKER_OPTS="-p 8000:8000"
|
Windows
Windows PowerShell: hab-web-app-builder
PS > | $env:HAB_DOCKER_OPTS = "-p 8000:8000"
|
Recall that the Studio runs in a Docker container. This environment variable configures port forwarding from port 8000 on your system to port 8000 in the container. You'll use this to access the web app (which runs in the Studio) from a browser on your host system.
Now enter the Studio.
Build the package.
Hab Studio
[1][default:/src:0]# | build
|
Start the web application. Replace learn-chef file you see with your origin name.
Hab Studio
[2][default:/src:0]# | hab svc load learn-chef/habitatize
|
The learn-chef/habitatize service was successfully loaded
|
From your browser, navigate to http://localhost:8000 or http://127.0.0.1:8000. You see this.
Choose a GIF or PNG file and then click Upload image. The project comes with a PNG file you can use. You see this.
When you're done, exit the Studio.
Hab Studio
[3][default:/src:0]# | exit
|
logout
|
You can leave your browser open for later.
2. Connect the web app to Builder
In practice, you might first run lint, unit, and integration tests on your app to ensure it passes basic quality gates.
But let's say that you're ready to share your app with others. In the Try Habitat module, you uploaded a Habitat package that you built from the Studio to Builder.
Here you connect your GitHub repo to Builder so that Builder can automatically rebuild your package for you.
Authorize Builder to access to your repo
When you ran through the Habitat demo, you set up the Habitat Builder GitHub app. There you had the option to grant Builder access to all of your repos or just select ones. If you chose just select repos, you need to add your repo to the list. Here's how:
- Sign into the Habitat Builder GitHub app.
- Click Configure.
- Select the GitHub org that contains your
hab-web-app-builder repo. - Enter your GitHub password if prompted.
Under Repository access, enter "hab-web-app-builder", select your repo, then click Save.
Connect Builder to your repo
- Navigate to Habitat Builder and sign in with your GitHub account.
- From the My Origins page, select your origin.
From the Packages tab, click Connect a plan file.
Select your GitHub organization and the hab-web-app-builder repo.
Although not required, we recommend you publish your package to Docker Hub.
Click the on switch and enter your Docker organization and repository.
Click Save Connection.
Configure your Docker Hub connection - click one your origin, and then the Integrations tab and enter your Docker organization and repository.
Click Save Account.
Update the package origin
Your plan needs to contain your origin so that the build system can use your signing key. Your origin uniquely identifies you as the upstream of the package.
Open ~/hab-web-app-builder/habitat/plan.sh and change the value of pkg_origin to match your origin.
Editor: ~/hab-web-app-builder/habitat/plan.sh
1
2
3
4
5
| pkg_name=habitatize
pkg_origin=YOUR_HAB_ORIGIN
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="0.1.0"
pkg_deps=( core/imagemagick ) |
Commit the change.
Terminal: ~/hab-web-app-builder
$ | git commit -a -m "Update origin"
|
Push your change up to GitHub.
Terminal: ~/hab-web-app-builder
| In practice, you would typically create a feature branch for your work instead of committing directly to master. |
Watch your package build automatically
From Builder, refresh the page. From the Build Jobs tab you see that Builder is building your package. The build job is similiar to what you previously did in the Studio.
The build takes a couple minutes to complete. You can click View the output to follow the log output as the package builds.
When the build completes, switch to the Versions tab. You see that a new version of your package is available.
The actual version number maps to the value of pkg_version in the plan.sh file.
See the results
If you connected the project to Docker Hub, you see that your image appears under three tags:
latestpkg_versionpkg_version-pkg_release
That's because you enabled these tags when you connected your repo to Builder.
Run this docker pull command to pull down your image. Replace learnchef/habitatize with your Docker Hub org and repo names.
Terminal: ~/hab-web-app-builder
$ | docker pull learnchef/habitatize
|
Run this docker run command to run it as a container. Again, replace learnchef/habitatize with your names.
Terminal: ~/hab-web-app-builder
$ | docker run -it -p 8000:8000 learnchef/habitatize
|
Refresh your browser. You see this.
You can interact with the app if you'd like to verify that it works. When you're finished, enter Ctrl+C to stop the Docker container.
You can also download and run your Habitat package directly from Builder from the Studio.
Start by running hab studio enter.
Terminal: ~/hab-web-app-builder
Then run this command to download and install your package.
Hab Studio
[1][default:/src:0]# | hab pkg install --channel unstable $HAB_ORIGIN/habitatize
|
You need to specify --channel unstable because you have not yet tagged a release as stable.
Then run this command to start it.
Hab Studio
[2][default:/src:0]# | hab svc load $HAB_ORIGIN/habitatize
|
Refresh your browser and you'll see the app. You can also run curl to access the app from the Studio. Remember you need to install curl first.
Hab Studio
[3][default:/src:0]# | hab pkg install core/curl -b
|
Hab Studio
[4][default:/src:0]# | curl localhost:8000
|
<html>
|
<head></head>
|
<body style='background-color: #87B09A;'>
|
<h1>Meme Green Machine!</h1>
|
|
<form action="/habitatized" method="POST" enctype="multipart/form-data">
|
<input type="file" name="file">
|
<input type="submit" value="Upload image">
|
</form>
|
</body>
|
</html>
|
Although the Git repo is named hab-web-app-builder, you specify habitatize because that's what the plan names the package. A Git repo can contain multiple plans.
Editor: ~/hab-web-app-builder/habitat/plan.sh
1
2
3
4
5
| pkg_name=habitatize
pkg_origin=YOUR_HAB_ORIGIN
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="0.1.0"
pkg_deps=( core/imagemagick ) |
Check your GitHub issues
At this point, your web application is not yet deployed to an environment where end users can access it. We'll cover that in a later module.
But your Habitat package is available on Builder and your Docker image is available on Docker Hub. (For learning purposes, your package and your image are publicly available to anyone, but you can set either to have private access only.)
Let's say you discover that someone's left you an issue in your GitHub repo.
Although not required, you can verify this for yourself. Remember, at this point you can bring up the app 3 different ways:
- Download the Docker image and then run it.
- From the Studio, build the app from source code and then run it.
- From the Studio, download the package from Builder and then run it.
You likely still have the Studio running (option 3).
You get this error when you specify a JPEG image. (The project comes with a JPEG file you can use.)
Exit the Studio before moving on to the next part.
Hab Studio
[5][default:/src:0]# | exit
|
logout
|
3. Get ImageMagick
The Build a Habitat package from source module presents a similar scenario, so we won't cover it at length here.
To summarize, the web app uses a utility called ImageMagick to generate the animated GIF files. Specifically, it uses the core/imagemagick package to install ImageMagick. The core/imagemagick package is not compiled with JPEG support, but you can create a similar plan that does compile ImageMagick to include JPEG support.
To start, create a fork of https://github.com/learn-chef/imagemagick.
Next, move back to your working directory, for example, your home directory.
Next, clone the hab-web-app-builder. Replace GITHUB_ORG with your name.
Terminal: ~
$ | git clone https://github.com/GITHUB_ORG/imagemagick.git
|
Move to the imagemagick directory.
For brevity, we'll omit building and verifying the package locally through the Studio. For now, open ~/imagemagick/plan.sh and change the value of pkg_origin to match your origin.
Editor: ~/imagemagick/plan.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
| pkg_name=imagemagick
pkg_origin=YOUR_HAB_ORIGIN
pkg_version=6.9.2-10
pkg_description="A software suite to create, edit, compose, or convert bitmap images."
pkg_upstream_url="http://imagemagick.org/"
pkg_license=('Apache2')
pkg_maintainer="The Habitat Maintainers <humans@habitat.sh>"
pkg_source=http://www.imagemagick.org/download/releases/ImageMagick-${pkg_version}.tar.xz
pkg_shasum=da2f6fba43d69f20ddb11783f13f77782b0b57783dde9cda39c9e5e733c2013c
pkg_deps=(core/libjpeg-turbo core/glibc core/zlib core/libpng core/xz core/gcc-libs)
pkg_build_deps=(core/zlib core/coreutils core/diffutils core/patch core/make core/gcc core/sed core/glibc core/pkg-config)
pkg_bin_dirs=(bin)
pkg_include_dirs=(include/ImageMagick-6)
pkg_lib_dirs=(lib)
pkg_dirname=ImageMagick-${pkg_version}
do_build() {
CC="gcc -std=gnu99" ./configure --prefix=$pkg_prefix --with-jpeg=yes
make
} |
This plan resembles the core/imagemagick plan. But notice that pkg_deps includes core/libjpeg-turbo and that the compilation flags in the do_build callback includes the --with-jpeg=yes argument. These attributes enable JPEG support in ImageMagick.
Commit your change.
Terminal: ~/imagemagick
$ | git commit -a -m "Update origin"
|
Push your change up to GitHub.
4. Connect ImageMagick to Builder
Here you follow a process similar to the one you used to connect your web application repo to Builder.
Authorize Builder to access to your repo
As you did before, if you granted only select access to the Habitat Builder GitHub app, you need to add your repo to the list. Here's how.
- Sign into the Habitat Builder GitHub app.
- Click Configure.
- Select the GitHub org that contains your
imagemagick repo. - Enter your GitHub password if prompted.
Under Repository access, enter "imagemagick", select your repo, then click Save.
Connect Builder to your repo
- Navigate to Habitat Builder and sign in with your GitHub account.
- From the My Origins page, select your origin.
From the Packages tab, click Connect a plan file.
Enter your GitHub organization and the imagemagick repo.
Unlike for your web application, do not publish your package to Docker Hub. Also, change the path to the plan file to plan.sh.
Click Save Connection.
You do not need to publish this project to Docker Hub because ImageMagick is a utility, much like curl or sed. It's not something you would likely want to deploy on its own container instance. You typically include a utility package as a dependency of another package, as you'll do shortly.
When an application's source code includes its Habitat plan, such as with the web application, you typically place the plan.sh file in the habitat directory. But in the case of the ImageMagick Habitat plan, plan.sh is the only file. The plan downloads the source code from imagemagick.org. Therefore, the plan is not stored in the habitat directory.
Build it from the website
For the web application, you saw how Builder can automatically build the package for you when you push a change to GitHub.
You can also initiate the build manually. In this case, a manual build is useful because you pushed a change to your GitHub repo before you connected the repo to Builder.
From the Build Jobs tab, click Build latest version.
The build takes a few minutes to complete. You can click View the output to follow the log output as the package builds.
When the build completes, switch to the Versions tab. You see that version 6.9.2-10 of your package is available.
5. Use your version of ImageMagick in the web app
At this point, you have two Builder packages – habitatize and imagemagick. If you're using Docker Hub, you also have a Docker image that contains the imagemagick package.
Your next step is to update the web application to use your imagemagick package and not core/imagemagick. Let's start by connecting the package to the app and ensuring it builds locally through the Studio.
Start by moving to the hab-web-app-builder directory.
Terminal: ~/imagemagick
$ | cd ~/hab-web-app-builder
|
Update the plan
In the plan file, notice that pkg_deps includes core/imagemagick.
Editor: ~/hab-web-app-builder/habitat/plan.sh
1
2
3
4
5
| pkg_name=habitatize
pkg_origin=YOUR_HAB_ORIGIN
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="0.1.0"
pkg_deps=( core/imagemagick ) |
Replace core/imagemagick with your package name. Also update pkg_version to 0.2.0. Here's an example.
Editor: ~/hab-web-app-builder/habitat/plan.sh
1
2
3
4
5
| pkg_name=habitatize
pkg_origin=learn-chef
pkg_scaffolding="core/scaffolding-ruby"
pkg_version="0.2.0"
pkg_deps=( learn-chef/imagemagick ) |
This example uses the learn-chef Habitat origin. Your origin will be unique.
For this project, we follow Semantic Versioning rules. We increment the MINOR version from 1 to 2 because we're adding backwards-compatible functionality.
Build the package
Enter the Studio.
Terminal: ~/hab-web-app-builder
Build the package.
Hab Studio
[1][default:/src:0]# | build
|
[...]
|
» Installing learn-chef/imagemagick from channel 'stable'
|
∅ The package does not have any versions in the specified channel.
|
∅ Did you intend to install one of the folowing instead?
|
∅ learn-chef/imagemagick/6.9.2-10/20171011151936 in channel bldr-826017590842802176
|
∅ learn-chef/imagemagick/6.9.2-10/20171011151936 in channel unstable
|
✗✗✗
|
✗✗✗ Package not found
|
✗✗✗
|
habitatize: WARN Could not find a suitable installed package for 'learn-chef/imagemagick'
|
ERROR: Resolving 'learn-chef/imagemagick' failed, should this be built first?
|
Notice the error. When you specify a runtime or build dependency, Habitat retrieves the latest package from the stable channel. In this example, Habitat was unable to find the latest stable version of the learn-chef/imagemagick package. That's because Builder tags each package with the unstable tag. You need to manually promote your package to stable once you have a build you want to release.
You'll promote your imagemagick package shortly. But first you want to verify that your version can properly convert JPEG files to animated GIFs.
Install imagemagick from the unstable channel
To pull in your imagemagick package, you need to manually install it into your Studio environment. To manually install a package from the unstable channel, you need to specify it by its fully-qualified name.
To get the fully-qualified name, run this hab pkg search command.
Hab Studio
[2][default:/src:0]# | hab pkg search $HAB_ORIGIN/imagemagick
|
learn-chef/imagemagick/6.9.2-10/20171012142806
|
Note the output you see. You can also get the fully-qualified name from the failed build output or from Builder.
Next, run hab pkg install to install your package. Replace the package name you see with yours.
Hab Studio
[3][default:/src:0]# | hab pkg install --channel unstable learn-chef/imagemagick/6.9.2-10/20171012142806
|
Build the package a second time
Let's try the build again. Run build a second time to build your web app.
Hab Studio
[4][default:/src:0]# | build
|
[...]
|
» Installing learn-chef/imagemagick from channel 'stable'
|
∅ The package does not have any versions in the specified channel.
|
∅ Did you intend to install one of the folowing instead?
|
∅ learn-chef/imagemagick/6.9.2-10/20171011151936 in channel bldr-826017590842802176
|
∅ learn-chef/imagemagick/6.9.2-10/20171012142806 in channel bldr-826716440583544832
|
∅ learn-chef/imagemagick/6.9.2-10/20171012142806 in channel unstable
|
✗✗✗
|
✗✗✗ Package not found
|
✗✗✗
|
habitatize: Resolved dependency 'learn-chef/imagemagick' to /hab/pkgs/learn-chef/imagemagick/6.9.2-10/20171012142806
|
[...]
|
This time, the build succeeds. The build process is able to resolve the dependency because it finds the package locally under /hab/pkgs.
Start the app
Next, run hab svc start to start the app. Replace the package name you see with yours.
Hab Studio
[4][default:/src:0]# | hab svc load ./results/learn-chef-habitatize-0.2.0-20171012164813-x86_64-linux.hart
|
hab-sup(MN): Supervisor starting learn-chef/habitatize/0.2.0/20171012164813. See the Supervisor output for more details.
|
Refresh your browser. Choose a JPEG file and then click Upload image. The project comes with a JPEG file you can use. You see this.
You can also choose a GIF or PNG image to verify you haven't broken existing functionality.
Great! You've successfully verified that your ImageMagick library works with the web app. Now exit the Studio.
Hab Studio
[5][default:/src:0]# | exit
|
logout
|
Promote the imagemagick package to stable
You're almost ready to commit your change to the imagemagick repo, push your change to GitHub, and have Builder automatically rebuild your package.
First, you need to promote your imagemagick package to the stable channel. Otherwise, the build job on Builder will fail as you saw when you tested your change locally.
In Try Habitat, you manually published your package to Builder and promoted it to the stable channel. You'll repeat a similar process here.
Repeat the hab pkg search command to verify the fully-qualified package name.
Terminal: ~/hab-web-app-builder
$ | hab pkg search $HAB_ORIGIN/imagemagicklearn-chef/imagemagick/6.9.2-10/20171012162836
|
As a verification step, run hab pkg channels to check that this package belongs to the unstable channel.
Terminal: ~/hab-web-app-builder
$ | hab pkg channels learn-chef/imagemagick/6.9.2-10/20171012162836» Retrieving channels for learn-chef/imagemagick/6.9.2-10/20171012162836bldr-826777091620331520unstable
|
Run hab pkg promote to promote your package to the stable channel. Replace the token and package name you see here with yours.
Terminal: ~/hab-web-app-builder
$ | hab pkg promote learn-chef/imagemagick/6.9.2-10/20171012162836 stable» Promoting learn-chef/imagemagick/6.9.2-10/20171012162836 to channel 'stable'✓ Promoted learn-chef/imagemagick/6.9.2-10/20171012162836
|
Run hab pkg channels a second time and you see that the stable tag appears in the output.
Terminal: ~/hab-web-app-builder
$ | hab pkg channels learn-chef/imagemagick/6.9.2-10/20171012162836 » Retrieving channels for learn-chef/imagemagick/6.9.2-10/20171012162836bldr-826777091620331520stableunstable
|
From Builder, navigate to your imagemagick project. From the Latest tab, you see that your package is tagged stable.
Commit your change and rebuild on Builder
You're ready to commit and push your change to GitHub.
First, commit your change.
Terminal: ~/hab-web-app-builder
$ | git commit -a -m "Use my ImageMagick package"
|
Now push your change up to GitHub.
Terminal: ~/hab-web-app-builder
From Builder, switch over to your habitatize project.
You see that the build started automatically. As before, you can view the output of the running build.
Verify the result
When the build completes, pull the updated image from Docker Hub and run the container. Remember to replace learnchef/habitatize with your names.
Terminal: ~/hab-web-app-builder
$ | docker pull learnchef/habitatize
|
Terminal: ~/hab-web-app-builder
$ | docker run -it -p 8000:8000 learnchef/habitatize
|
As an optional step, you can verify the app in your browser. You see that the web app that's running in the Docker container successfully loads JPEG images.
6. Rebuild the habitatize package automatically
So far, you've pushed changes to each of the habitatize and imagemagick repositories and watched Builder rebuild your packages.
When Builder builds your package, it also rebuilds any packages that depend on that package. For example, the habitatize package depends on imagemagick:
habitatize- └─
imagemagick
When imagemagick builds, habitatize is also rebuilt.
As a second example, if a security vulnerability is discovered in any of the "core" packages, such as core/openssl, any package that depends on core/openssl is automatically rebuilt after core/openssl is rebuilt to address the vulnerability. This enables your packages to stay current with the latest releases without the need to take action.
Builder also publishes a Docker image if any associated Builder project in the dependency chain is set up for Docker Hub.
Let's push up one more small change to see a dependent rebuild in action.
Consider the do_build callback in the plan.sh file for imagemagick.
Editor: ~/imagemagick/plan.sh
1
2
3
4
5
6
| [...]
do_build() {
CC="gcc -std=gnu99" ./configure --prefix=$pkg_prefix --with-jpeg=yes
make
} |
The compliation flags, CC, contains --with-jpeg=yes. You added that earlier when you modified ImageMagick to support JPEG images.
You later discover that the --with-jpeg flag isn't required. Recall that your imagemagick plan includes core/libjpeg-turbo in its dependency list. When the core/libjpeg-turbo package is installed, a number of C header files are written to disk. The ImageMagick build script automatically detects the presence of these files and builds in JPEG support automatically.
Modify ~/imagemagick/plan.sh to omit the --with-jpeg flag in do_build, like this.
Editor: ~/imagemagick/plan.sh
1
2
3
4
5
6
| [...]
do_build() {
CC="gcc -std=gnu99" ./configure --prefix=$pkg_prefix
make
} |
In practice, you would likely test things out before you commit the change. For brevity, let's go ahead and commit the change and push up your master branch to GitHub.
Terminal: ~/imagemagick
$ | git commit -a -m "Remove --with-jpeg flag"
|
Return to your imagemagick project on Builder. Refresh the page and you'll see that a build is in progress. Follow the log to watch your package build.
When the build completes, navigate to your habitatize project. You see that a build was automatically started because the dependent package was just rebuilt.
Although not required, you can pull down the Docker image one last time to verify things. Remember to replace learnchef/habitatize with your names.
Terminal: ~/imagemagick
$ | docker pull learnchef/habitatize
|
Terminal: ~/imagemagick
$ | docker run -it -p 8000:8000 learnchef/habitatize
|
The web app will continue to function as it did previously, including with JPEG files.
Congratulations! In this module, you connected two projects to Builder. Builder is a service that
