Jenkins and sbt

Posted on Wednesday, January 29, 2014


This guide will go over installing the sbt plugin for Jenkins.  This guide assumes you Jenkins already installed and running and has the git plugin. 

If you do not have Jenkins installed I wrote up a guide to show you how at http://www.whiteboardcoder.com/2014/01/install-jenkins-ubuntu.html

If you do not have the git plugin for Jenkins installed I wrote up a guide on that too at http://www.whiteboardcoder.com/2014/01/git-with-jenkins.html



Install sbt-launch.jar


Information for this part was taken from the sbt install guide at http://www.scala-sbt.org/release/docs/Getting-Started/Setup.html [2]

Run the following to download and install sbt-launch.jar


> wget http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.1/sbt-launch.jar
> sudo mv sbt-launch.jar /bin


Create a run script for it at /bin/sbt


> sudo vi /bin/sbt


And place in it


SBT_OPTS="-Xms512M -Xmx1536M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M"
java $SBT_OPTS -jar `dirname $0`/sbt-launch.jar "$@"


Make it executable


> sudo chmod a+x /bin/sbt



Run it as a test


> sbt


Looks like it downloads a lot of libraries the first time




Install plug in


Open your Jenkins install in a web browser.  Mine happens to be located at http://192.168.0.7:8080/

After you login click on Manage Jenkins




Click on Manage Plugins





Click on the Available Tab





In the filter enter "sbt"
Then select "sbt Plugin" and finally click on "Install without restart"






Plug-in installation starts





After the plug-in is successfully installed click on "Go back to the top page"






Configure Jenkins to use sbt launch jars


Click on Manage Jenkins

 

Click on Configure system




Scroll down to sbt and click Add


 


Give it a name  (I named it after the sbt-launch version 0.13.1)
And set the path to the sbt-launch.jar location.

Click Save








Add build step to your project

Click on New Job


 

Give the job a name
Select "Build a free-style software project"
Click OK



Scroll down and click "add build step"
And select "Build using sbt"




Enter your action and click Save

 



Real example


I am very familiar with Hudson using ant builds from a subversion repository (Hudson was Jenkins former name).   I am not yet very familiar with scala or sbt.   So I want to make a simple example here that uses git, scala code, sbt, and Jenkins to build and deploy some code.

Chances are this will be a very longwinded example so I am warning you right now it will be long….


Set up the folder structure



From this page

sbt uses the same directory structure as Maven for source files by default.



Here is the structure

src/
  main/
    resources/
       <files to include in main jar here>
    scala/
       <main Scala sources>
    java/
       <main Java sources>
  test/
    resources
       <files to include in test jar here>
    scala/
       <test Scala sources>
    java/
       <test Java sources>


I am only concerned about scala code so my directory structure will be.

src/
  main/
    resources/
    scala/


I am going to make a test directory and within that directory create the above directory structure.  Here is the linux command line code to make that happen.


> cd
> mkdir test
> cd test
> mkdir -p src/main/resources
> mkdir src/main/scala


I am going to put my code in the package com.whiteboardcoder.hello so I need to make some more directories.



> mkdir -p src/main/scala/com/whiteboardcoder/hello


Create the code.


> vi src/main/scala/com/whiteboardcoder/hello/hw.scala


Here is the code

package com.whiteboardcoder.hello

object hw {
  def main(args: Array[String]){
    println("Hello World!")
  }
}

Create the build.sbt file


> vi build.sbt


Here is the basic build.sbt code


name:= "My Hello World Program"

version := "1.0"

scalaVersion := "2.10.3"



Now, as a test, compile it


> sbt compile




This will create a project and a target folder.  It will place the compiled code at

target/scala-2.10/classes/com/whiteboardcoder/hello/hw.class



According to http://www.scala-sbt.org/release/docs/Getting-Started/Running.html [4] sbt has the following commands

clean                - Deletes all generated files in the target directory
compile            - Compiles the main sources
test                  - Compiles and runs all tests
console            - Starts the Scala Interpreter
run                   - Runs the main class for the project
package           - Creates a jar file containing al the files in
                        src/main/resources and the classes compiled from
                        src/main/scala and src/main/java
help                 - Displays detailed help
reload              - Reloads the build definition


Now, as a test, run… clean compile package


> sbt clean compile package


This created a jar file named my-hello-world-program_2.10-1.0.jar at
target/scala-2.10/my-hello-world-program_2.10-1.0.jar

This name came from the build.sbt file the "name" setting

This makes me think…. I would like to set the main class for the jar package (Set in the Manifest)


Create jar with mainClass


Edit the build.sbt file


> vi build.sbt


Add the mainClass setting


name:= "My Hello World Program"

version := "1.0"

scalaVersion := "2.10.3"

mainClass in Compile := Some("com.whiteboardcoder.hello.hw")



As a quick test run it


> sbt clean compile run


The run used the mainClass




Now do a package (create a jar)


> sbt clean compile package


Try to run the compile jar


> java -jar target/scala-2.10/my-hello-world-program_2.10-1.0.jar


I get the following error

Exception in thread "main" java.lang.NoClassDefFoundError: scala/Predef$

I am missing scala-library.jar….

After looking around a bit it seems that sbt out of the box does not have a good way to combine all the necessary dependent jar files into one jar file.  But there are several other tools out there to help with this.

I found sbt-assembly at https://github.com/sbt/sbt-assembly [5]

This tool can be loaded automatically in your sbt build.  sbt uses Apache Ivy http://ant.apache.org/ivy/ [6].  I am not yet familiar with all that Ivy can do, but in this case it can download and install an sbt plugin in with one line of script.


Create a new file at project/plugins.sbt


> vi project.plugins.sbt


And add the following line to it.


addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.10.2")


Nothing more is needed as the default Ivy Repository has sbt-assembly in it http://www.scala-sbt.org/release/docs/Community/Community-Plugins.html [7], otherwise I think you would have to append to the resolvers to tell sbt where to find the plugin.




Edit build.sbt


> vi build.sbt


With the following highlighted changes


import AssemblyKeys._

assemblySettings

name:= "My Hello World Program"

version := "1.0"

scalaVersion := "2.10.3"

jarName in assembly := "hello.jar"

mainClass in (Compile, assembly) := Some("com.whiteboardcoder.hello.hw")


This will import and set the assembly setting keys.  Then set the name of the jar to hello.jar

Now run, the first time you run this it will download the sbt-assembly plug-in


> sbt clean compile assembly



Try to run the compiled jar


> java -jar target/scala-2.10/hello.jar




That worked.

If you look in the jar file


> jar tf target/scala-2.10/hello.jar


You will see that it has all the scala libraries needed to run this program have been included



This makes the jar file about 13 MiB in size


Clean up and upload to a git server


I am going to first add a .gitignore file, there are some folders and files we do not want to add to a git repository.


> vi .gitignore


And place the following in it


/target
/project/project
/project/target



Now initialize a git repository


> git init
> git add .
> git commit –m ‘initial commit’


To check which files are ignored by git run the following command


> git status --ignored






Remote SSH server


I am not going to go over how to set up a remote git server, I am going to assume you have a git server running somewhere.  (don't forget SSH keys)

If you need help with that you can refer to a how to I wrote before on the subject at http://www.whiteboardcoder.com/2012/08/installing-git-server-on-ubuntu-1204.html

On my git server I need to set up a bare bones repository to push to 

From the location where you keep your repositories run the following


> mkdir test-project.git
> cd test-project.git
> git --bare init



Push to git server


Back to the local files…  Add an origin to the local git repository and push it up.

Assuming your git repo can be accessed via SSH to git@example.com:/opt/git/test-project.git
Change this to fit  your situation


> git remote add origin git@example.com:/opt/git/test-project.git
> git push origin master


Ok so at this point you should have pushed this sbt project up to a remote git repository.


Now back to Jenkins!  Let's get Jenkins talking to the remote git server downloading it and running a build.



Jenkins and SSH keys


If you have not set up ssh keys for the Jenkins user on your system you need to create those now.
 In my set up the Jenkins home directory is at /var/lib/jenkins

Switch over to the Jenkins user and create its ssh keys


> sudo su jenkins
> cd
> ssh-keygen -t rsa -b 2048


Copy the id_rsa.pub key to obtain it you could run the following command



> cat .ssh/id_rsa.pub


Append this id_rsa.pub to the authorized_keys file in the .ssh directory of the git user on your git remote repository.





Open github repository


Create a new job to use git repository


Click on New Job



Name it Test
Select "Build a free-style software project"
Click OK




Select "Git" then enter the github remote repository

ssh://git@example.com/opt/git/test-project.git




Click Save




Click Build now



The Build was successful




This is a very simple deploy, all it does is download from the remote git repository.




Edit job to run sbt script


From the newly made job click Configure




Scroll down to "Add build step" and select Build using sbt



For this action I set it to
"clean compile assembly run"



Click Save



Click on "Build Now" again




After it completes click on the last build




Click on the Console Output



Here you will see the entire console output  (this first go around it takes a while as it needs to download some files)

Scrolling to the bottom you can see that the jar file was created at
/var/lib/jenkins/jobs/Test/workspace/target/scala-2.10/hello.jar
And the program was run resulting in the "Hello World" output.





I know this is not a complete deploy solutions but it should be enough to get you started.


References
[1]        SBT plugin
                Accessed 01/2014
[2]        SBT setup
                Accessed 01/2014
[3]        SBT Directory Structure
                Accessed 01/2014
[4]        SBT Running
                Accessed 01/2014
[5]        abt-assembly
            https://github.com/sbt/sbt-assembly
                Accessed 01/2014
[6]        Apache Ivy
            http://ant.apache.org/ivy/
                Accessed 01/2014
[7]        Community Ivy Repository¶
                Accessed 01/2014


No comments:

Post a Comment