Search This Blog

Saturday, January 28, 2012

SBT ~ Scala's Simple Build Tool Series, Part 3. SBT Default Settings


In the previous article, I explained what an SBT Setting is, and provided a list of Settings provided by SBT upon installation. In this article, I am going to describe some of the defaults provided by SBT out of the box.

SBT Default Settings and Scopes.
SBT's Default Settings are defined in the Defaults.scala source file. There are two configuration scopes where settings are defined, the GlobalScope configuration scope, and the ThisProject configuration scope (the project to be built, that is, YOUR project). Settings cascade, that is, each ScopeAxis (Global, Project, Configuration, or Task), falls back on another ScopeAxis if a setting is requested from it that was not defined in the scope of the task. In this way, SBT build definitions, which are sequences of initialized settings, are similar to Cascading Style Sheets, with lower-level ScopeAxis, like Task Scopes, overriding the settings of the ScopeAxis that they extend from (using the extend method).


At the top level, is the GlobalScope. You could use this scope to define things like yourself as an author or contributor, add the ENSIME plugin to all your projects, etc. All other scope definitions come from this Global root. In Defaults.scala, this is used to set defaults for many of the 220+ keys available in SBT. A consequence of this is that if you do not set a setting to what you want, SBT will set it to a default setting for you. An example, the name settingKey. Fire up SBT inside an empty project and don't create a name entry in the .sbt build definition file. Type inspect name at the prompt. You'll get something like the following:


> inspect name
[info] Setting: java.lang.String = default-42f6d6
[info] Description:
[info] Project name.
[info] Provided by:
[info] {file:/Users/jackviers/Library/Developer/sbtProj/}default-42f6d6/*:name
[info] Dependencies:
[info] *:this-project
[info] Reverse dependencies:
[info] *:project-info
[info] *:normalized-name
[info] *:on-load-message
[info] *:description
[info] Delegates:
[info] *:name
[info] {.}/*:name
[info] */*:name


Fig. 1. Result of not setting name in build using sbt.

Note that the name setting is a Setting[String] with the value of default-42f6d6. This is a great feature. It means that for your average build, you don't need to know a ton of SBT settings, because SBT will fill them in for you. It also means that, if you wish, you may override or add to existing SBT settings at almost any level of your build.





Important SBT Default Settings
The following is a list of prevalent default settings, with their ScopeAxis defaults where provided. You can find them in the SBT source code, mostly in the Defaults.scala file, or by starting sbt in interactive mode and using the inspect command.



  1. name, ThisProject, project id (default+ hash)
  2. organization, ThisProject, <<= name(StringUtilities.normalize)
  3. credentials, Global, Nil
  4. taskTemporaryDirectory, Global, Some system level temporary directory
  5. watchSources, ThisProject, Nil
  6. sourceDirectory, ThisProject, src
  7. scalaSource, ThisProject, sourceDirectory / "scala"
  8. javaSource, ThisProject, sourceDirectory / "java"
  9. compileOrder, Global, CompileOrder.Mixed
  10. javacOptions, Global, Nil
  11. scalacOptions, Global, Nil
Again, you can override any of the 220+ settings however you would like, but SBT, in most cases, will define them for you.

In the next article I will discuss the various tasks available out of the box with SBT.







Monday, January 16, 2012

SBT ~ Scala's Simple Build Tool Series, Part 2. SBT Settings

In the previous article, I told you about Scala's Simple Build Tool (SBT), how to install and set up a simple Scala project, and how to use SBT to build and run that project. In this article, I'm going to explain what an SBT Setting is, and provide a list of Settings provided by SBT upon installation.


SBT Settings
SBT Setting objects are made up of three parts: Scope, name, and value. They are represented in the SBT api code as SettingKey traits. Settings are typed generically, and written as SettingKey[T]. The SettingKey trait type, T, defines the type of the SettingKey. For example, if the Setting is a String, the type of the SettingKey representing the Setting is String, and the object is written as SettingKey[String]. In practice, you won't see a SettingKey[String], but the example is there for you to realize that when you look at a SettingKey[T], the SettingKey is always typed to type T, where T is some trait, object, or class name, like String.


SBT Scopes
 Scopes are objects containing the names and values of keys within a scope, or section, of the project. They give a setting a context to live in. This allows you to have different settings for your test build than in your src build; a subproject within your build than in your main project of your build; or in the compile task than in the documentation task of your build. Scopes are defined by their ScopeAxis. There are three types of ScopeAxis: typed by Project, typed by Task, and typed by Configuration. The built-in configurations are a subset of those provided by Maven Dependency Scopes: Compile, Run, Package, Document, and Test. Project ScopeAxes refer to the current project. Task ScopeAxes refer to a Task. There is also a Global ScopeAxes, which sets the setting for all axes.


SBT Keys
The name and value of a setting are stored in scopes in a value trait called AttributeKey. The attribute key is typed to contain the type of value stored in the value part of the setting, and defined as AttributeKey[T]. So if a Setting has a string value, the AttributeKey type, T, is String and it is written as AttributeKey[String]. The name part of the Setting is the AttributeKey[T].label member of the AttributeKey[T] trait. The value is only available after evaluating the key within the given scope, and is returned by the AttributeKey[T].evaluate method.


Defining Settings
SBT Settings can be defined in a .sbt build definition file, a *.scala source file under the SBT project directory, or in the sbt users' home .sbt definition file, which defines global default settings for all projects built by sbt by a given user. To explicitly define a Setting within a Scope, you use the SettingKey[T].in method. For example, name in Compile := "My CompileName" sets the name AttributeKey[String] in the Compile configuration Scope.


Available Settings 
Here's a list of the over 220 Setting/Task/AttributeKeys (SBT SETTINGS), along with their descriptions, sbt command-line alias, and types. It comes directly from the Keys.scala SBT source file. I've sub-sectioned them by purpose, again, coming directly from the source file. Remember, this is just a list of the available settings. To define a full setting, you must give it a scope (project is the default in .sbt build definition files) and a value (like "MyProjectName") that matches the type of the available key. 


TL;DR
This list is not intended to be read linearly. Use the following shortcut links to skip to a section when looking for a particular type of SBT setting, its type, and description. An alternative is to use the browser search to look for a particular setting name, type, and description. In the next article, I am going to talk about some of the default settings that come out of the box with SBT.





Logging Settings
Format (key name, key type, sbt command-line key name, description)
  • logLevel, SettingKey[Level.value], log-level, The amount of logging sent to the screen.
  • persistLogLevel, SettingKey[Level.value], persist-log-level, The amount of logging sent to a file for persistence.
  • traceLevel, SettingKey[Int], trace-level, The amount of stack-trace to be displayed.
  • persistTraceLevel, SettingKey[Int], persist-trace-level, The amount of stack-trace to be persisted (to a file).
  • showSuccess, SettingKey[Boolean], show-success, If true, displays a success message after running a command successfully.
  • showTiming, SettingKey[Boolean], show-timing, If true, the command success message includes the completion time.
  • timingFormat, SettingKey[java.text.DateFormat], timing-format, The format used for displaying the completion time.
  • extraLoggers, SettingKey[PartialFunction[ScopedKey[_], Seq[AbstractLogger]]], extra-loggers, A function that provides additional loggers for a given setting.
  • logManager, SettingKey[LogManager], log-manager, The log manager, which creates Loggers for different contexts (Configurations).
  • logBuffered, SettingKey[Boolean], log-buffered, True if logging should be buffered until work completes.

Project Settings
Format (key name, key type, sbt command-line key name, description)
  • projectCommand, AttributeKey[Boolean], project-command, Marks Commands that were registered for the current Project.
  • sessionSettings, AttributeKey[SessionSettings], session-settings, Tracks current build, project, and setting modifications.
  • stateBuildStructure, AttributeKey[Load.BuildStructure], build-structure, Data structure containing all information about the build definition.
  • buildStructure, TaskKey[Load.BuildStructure], build-structure, Provides access to the build structure, settings, and streams manager.
  • loadedBuild, SettingKey[Load.LoadedBuild], loaded-build, Provides access to the loaded project structure. This is the information available before, settings are evaluated.
  • buildDependencies, SettingKey[BuildDependencies], build-dependencies, Definitive source of inter-project dependencies for compilation and dependency management. This is populated by default by the dependencies declared on Project instances, but may be modified. The main restriction is that new builds may not be introduced.
  • appConfiguration, SettingKey[xsbti.AppConfiguration], app-configuration, Provides access to the launched sbt configuration, including the ScalaProvider, Launcher, and GlobalLock.
  • thisProject, SettingKey[ResolvedProject], this-project, Provides the current project for the referencing scope.
  • thisProjectRef, SettingKey[ProjectRef], this-project-ref, Provides a fully-resolved reference to the current project for the referencing scope.
  • configuration, SettingKey[Configuration], configuration, Provides the current configuration of the referencing scope.
  • commands, SettingKey[Seq[Command]], commands, Defines commands to be registered when this project or build is the currently selected project or build.
  • initialize, SettingKey[Unit], initialize, A convenience setting for performing side-effects during initialization.
  • onLoad, SettingKey[PartialFunction[State, State]], on-load, Transformation to apply to the build state when the build is loaded.
  • onUnload, SettingKey[PartialFunction[State, State]], on-unload, Transformation to apply to the build state when the build is unloaded.
  • onLoadMessage, SettingKey[String], on-load-message, Message to display when the project is loaded.
  • transformState, AttributeKey[PartialFunction[State, State]], transform-state, State transformation to apply after tasks run.
  • onComplete, SettingKey[Function0[Unit]], on-complete, Hook to run when task-evaluation completes. The type of this setting is subject to change, pending the resolution of SI-2915.

Command Settings
Format (key name, key type, sbt command-line key name, description)
  • globalLogging, AttributeKey[GlobalLogging], global-logging, Provides a global Logger, including command logging.
  • historyPath, SettingKey[Option[File]], history, The location where the command line history is persisted.
  • shellPrompt, SettingKey[PartialFunction[State, String]], shell-prompt, The function that constructs the command propmt form the current build state.
  • analysis, AttributeKey[inc.Analysis], analysis, Analysis of compilation, including dependencies and generated outputs.
  • watch, SettingKey[Watched], watch, Continuous execution configuration.
  • pollInterval, SettingKey[Int], poll-interval, Interval between checks for modified sources by the continuous execution command.
  • watchSources, TaksKey[Seq(File)], watch-sources, Defines the sources in this project for continuos execution to watch for changes.
  • watchTransitiveSources, TaskKey[Seq[File]], watch-transitive-sources, Defines the sources in all projects for continuous execution to watch.
  • watchingMessage, SettingKey[PartialFunction1[WatchState,String]], watching-message, The message to show when triggered execution waits for sources to change.
  • triggeredMessage, SettingKey[PartialFunction[WatchState, String]], triggered-message, The message to show before triggered execution executes an action after sources change.
Path Settings
Format (key name, key type, sbt command-line key name, description)
  • baseDirectory, SettingKey[File], base-directory, The base directory. Depending on the scope this is the base directory for the build, project, configuration, or task.
  • globalBaseDirectory, AttributeKey[File], global-base-direcotry, The base directory for global sbt configuration and staging.
  • target, SettingKey[File], target, Main directory for files generated by the build.
  • crossTarget, SettingKey[File], cross-target, Main directory for files generated by the build that are cross-built.
Source Path Settings
Format (key name, key type, sbt command-line key name, description)
  • sourceDirectory, SettingKey[File], source-directory, Default directory containing sources.
  • sourceManaged, SettingKey[File], source-managed, Default directory for sources generated by the build.
  • scalaSource, SettingKey[File], scala-source, Default Scala source directory.
  • javaSource, SettingKey[File], java-source, Default Java source directory.
  • sourceDirectories, SettingKey[Seq[File]], source-directories, List all source directories, both managed and unmanaged.
  • unmanagedSourceDirectories, SettingKey[Seq[File]], unmanaged-source-directories, Unmanaged source directories, which contain manually created source files.
  • unmanagedSources, TaskKey[Seq[File]], unmanaged-sources, Unmanaged sources, which are manually created.
  • managedSourceDirectories, SettingKey[Seq[File]], managed-source-directories, Managed source directories, which contain sources generated by the build.
  • mangedSources, TaskKey[Seq[File]], managed-sources, Sources generated by the build.
  • managedSources, TaskKey[Seq[File], managed-sources, Sources generated by the build.
  • sources, TaskKey[Seq[File]], sources, All sources, both managed and unmanaged.
Filter Settings
Format (key name, key type, sbt command-line key name, description)
    • includeFilter, SettingKey[FileFIlter], include-filter, Filter for including sources and resources files from default directories.
    • excludeFilter, SettingKey[FileFilter], exclude-filter, Filter for excluding sources and resources files from default directories.


    Resource Path Settings
    Format (key name, key type, sbt command-line key name, description)
    • resourceDirectory, SettingKey[File], resource-directory, Default unmanaged resource directory, used for user-defined resources.
    • resourceManaged, SettingKey[File], resource-managed, Default managed resource directory, used when generating resources.
    • unmanagedResourceDirectories, SettingKey[Seq[File]], unmanaged-resource-directories, Unmanaged resource directories, containing resources manually created by the user.
    • unmanagedResources, TaskKey[Seq[File]], unmanaged-resources, Unmanaged resources which are manually created.
    • managedResourceDirectories, SettingKey[Seq[File]], managed-resource-directories, List of managed resource directories.
    • managedResources, TaskKey[Seq[File]], managed-resources, Resources generated by the build.
    • resourceDirectories, SettingKey[Seq[File]], resource-directories, List of all resource directories, both managed and unmanaged.
    • resources, TaskKey[Seq[File]], resources, All resource files, both managed and unmanaged.
    Output Path Settings
    Format (key name, key type, sbt command-line key name, description)
    • classDirectory, SettingKey[File], class-directory, Directory for compiled classes and copied resources.
    • cacheDirectory, SettingKey[File], cache-directory, Directory used for caching task data.
    • cleanFiles, SettingKey[Seq[File]], clean-files, The files to recursively delete during a clean.
    • cleanKeepFiles, SettingKey[Seq[File]], clean-keep-files, Files to keep during a clean.
    • crossPaths, SettingKey[Boolean], cross-paths, If true, enables, cross paths, which distinguish output directories for cross-building (cross-building means across scala versions)
    • taskTempDirectory, SettingKey[File], task-temporary-directory, Directory used for temporary files for tasks that is deleted after each task execution.
    Generators Settings
    Format (key name, key type, sbt command-line key name, description)
      • sourceGenerators, SettingKey[Seq[Task[Seq[File]]]], source-generators, List of tasks that generate sources.
      • resourceGenerators, SettingKey[Seq[Task[Seq[File]]]], resource-generators, List of tasks that generate resources.
      Compile and Doc Settings
      Format (key name, key type, sbt command-line key name, description)
      • autoCompilerPlugins, SettingKey[Boolean], auto-compiler-plugins, If true, enables automatically generating -Xplugin arguments to the compiler based on the classpath for the CompilerPlugin.name configuration.
      • maxErrors, SettingKey[Int], max-errors, The maximum number of errors, such as compile errors, to list.
      • scalacOptions, TaskKey[Seq[String]], scalac-options, Options for the Scala compiler.
      • javacOptions, TaskKey[Seq[String]], javac-options, Options for the Java compiler.
      • compileOrder, SettingKey[CompileOrder.Value], compile-order, Configures the order in which Java and sources within a single compilation are compiled. Valid values are, JavaThenScala, ScalaThenJava, or Mixed.
      • initialCOmmands, SettingKey[String], initialcommands, Initial commands to execute when starting up the Scala interpreter.
      • cleanupCommands, SettingKey[String], cleanup-commands, Commands to execute before the Scala interpreter exits.
      • compileInputs, TaskKey[Compiler.Inputs], compile-inputs, Collects all inputs needed for compilation.
      • scalaHome, SettingKey[Option[File]], scala-home, If Some, defines the local Scala installation to use for compilation, running, and testing.
      • scalaInstance, TaskKey[ScalaInstance], scala-instance, Defines the Scala instance to use for compilation, running, and testing.
      • scalaVersion, SettingKey[String], scala-version, The version of Scala used for building.
      • scalaBinaryVersion, SettingKey[String], scala-binary-version, The Scala version substring describing binary compatibility.
      • crossScalaVersions, SettingKey[Seq[String]], cross-scala-versions, The versions of Scala used when cross-building
      • classpathOptions, SettingKey[ClasspathOptions], classpath-options, Configures handling of Scala classpaths.
      • definedSbtPlugins, TaskKey[Set[String]], defined-sbt-plugins, The set of names of Plugin implementations defined by this project.
      • sbtPlugin, SettingKey[Boolean], sbt-plugin, If true, enables adding sbt as a dependency and auto-generation of the plugin descriptor file.
      • clean, TaskKey[Unit], clean, Deletes files produced by the build, such as generated sources, compiled classes, and task caches.
      • console, TaskKey[Unit], console, Starts the Scala interpreter with the project classes on the classpath.
      • consoleQuick, TaskKey[Unit], console-quick, Starts the Scala interpreter with the project dependencies on the classpath.
      • consoleProject, TaskKey[Unit], console-project, Starts the Scala interpreter with the sbt and the build definition on the classpath and useful imports.
      • compile, TaskKey[Analysis], compile, Compiles sources.
      • compilers, TaskKey[Compiler.Compilers], compilers, Defines the Scala and Java compilers to use for compilation.
      • compileIncSetup, TaskKey[Compiler.IncSetup], inc-compile-setup, Configurations aspects of incremental compilation.
      • definesClass, TaskKey[DefinesClass], defines-class, Internal use: provides a function that determines whether the provided file contains a given class.
      • doc, TaskKey[File], doc, Generates API documentation.
      • copyResources, TaskKey[Seq[(File, File)]], copy-resources, Copies resources to the output directory.
      • aggregate, SettingKey[Aggregation], aggregate, Configures task aggregation.
      Package Settings
      Format (key name, key type, sbt command-line key name, description)
      • packageBin, TaskKey[File], package-bin, Produces a main artifact, such as a binary jar.
      • package, TaskKey[File], package, Produces the main artifact, such as a binary jar. This is typically an alias for the task that actually does the packaging.
      • packageDoc, TaskKey[File], package-doc, Produces a documentation artifact, such as a jar containing API documentation.
      • packageSrc, TaskKey[File], package-src, Produces a source artifact, such as a jar containing sources and resources.
      • packageOptions, TaskKey[Seq[PackageOption]], package-options, Options for packaging.
      • packageConfiguration, TaskKey[Package.Configuration], package-configuration, Collects all inputs needed for packaging.
      • artifactPath, SettingKey[File], artifact-path, The location of a generated artifact.
      • artifact, SettingKey[Artifact], artifact, Describes an artifact.
      • artifactClassifier, SettingKey[Option[String]], artifact-classifier, Sets the classifier used by the default artifact definition.
      • artifactName, SettingKey[PartialFunction1[Tuple3[String, ModuleID, Artifact], String], artifact-name, Function that produces the artifact name from its definition.
      • mappings, TaskKey[Seq[Pair[File, String]]], mappings, Defines the mappings from a file to a path, used by packaging, for example.
      • fileMappings, TaskKey[Seq[Pair[File, File]]], file-mappings, Defines the mappings from a file to a file, used for copying files, for example.
      Run Settings
      Format (key name, key type, sbt command-line key name, description)
      • selectMainClass, TaskKey[Option[String]], select-main-class, Selects the main class to  run.
      • mainClass, TaskKey[Option[String]], main-class, Defines the main class for packaging or running.
      • run, InputKey[Unit], run, Runs a main class, passing along arguments provieded on the command line.
      • runMain, InputKey[Unit], run-main, Runs the main class selected by the first argument, passing the remaining arguments to the main method.
      • discoveredMainClasses, TaskKey[Seq[String]], discovered-main-classes, Auto-detects main classes.
      • runner, TaskKey[ScalaRun], runner, Implementation used to run a main class.
      • trapExit, SettingKey[Boolean], trap-exit, If true, enables exit trapping and thread management for 'run'-like tasks. This is currently only suitable for serially-executed 'run'-like tasks.
      • fork, SettingKey[Boolean], fork, If true, forks a new JVM when running. If false, runs in the same JVM as the build.
      • outputStrategy, SettingKey[Option[sbt.OutputStrategy]], output-strategy, Selects how to log output when running a main class.
      • connectInput, SettingKey[Boolean], connect-input, If true, connectsstandard input when running a main class forked.
      • javaHome, SettingKey[Option[File]], java-home, Selects the Java installation used for compiling and forking. If None, uses the java installation running the build.
      • javaOptions, TaskKey[Seq[String]], java-options, Options passed to a new JVM when forking.
      Test Settings
      Format (key name, key type, sbt command-line key name, description)
      • testLoader, TaskKey[ClassLoader], test-loader, Provides the class loader used for testing.
      • loadedTestFrameworks, TaskKey[Map[TestFramework, Framework]], loaded-test-frameworks, Loads Framework definitions from the test loader.
      • definedTests, TaskKey[Seq[TestDefinition]], defined-tests, Provides the list of defined tests.
      • definedTestNames, TaskKey[Seq[String]], defined-test-names, Provides the set of defined test names.
      • executeTests, taskKey[Tests.Output], execute-tests, Provies the list of defined tests.
      • test, TaskKey[Unit], test, Executes all tests.
      • testOnly, InputKey[Unit], test-only, Executes teh tests provided as arguments or all tests if no arguments are provided.
      • testOptions, TaskKey[Seq[TestOption]], Options for running tests.
      • testFrameworks, SettingKey[Seq[TestFramework]], test-frameworks, Registered, although not necessarily present, test frameworks.
      • testListeners, TaskKey[Seq[TestReportListener]], test-listeners, Defines test listeners.
      • testExecution, TaskKey[Tests.Execution], test-execution, Settings controlling test execution.
      • isModule, AttributeKey[Boolean], is-module, True if the target is a module.
      Classpath/Dependency Management Settings
      Format (key name, key type, sbt command-line key name, description)

      • name, SettingKey[String], name, Project name.
      • normalizedName, SettingKey[String], normalized-name, Project name transformed from mixed case and spaces to lowercase and dash-separated.
      • description, SettingKey[String], description, Project description.
      • homepage, SettingKey[Option[URL]], homepage, Project homepage.
      • startYear, SettingKey[Option[Int]], start-year, Year in which the project started.
      • licenses, SettingKey[Seq[Pair[String,URL]]], licenses, Project licenses as (name, url) pairs.
      • organization, SettingKey[String], organization, Organization/group ID.
      • organizationName, SettingKey[String], organization-name, Organization full/formal name.
      • organizationHomepage, SettingKey[Option[URL]], organization-homepage, Organization homepage.
      • projectInfo, SettingKey[ModuleInfo], project-info, Addition project information like formal name, homepage, licenses, etc.
      • defaultConfiguration, SettingKey[Option[Configuration]], default-configuration, Defines the configuration used when none is specified for a dependency.
      • defaultConfigurationMapping, SettingKey[String], default-configuration-mapping, Defines the mapping used for a simple, unmapped configuration definition.
      • products, TaskKey[Seq[File]], products, Build products that get packaged.
      • productDirectories, TaskKey[Seq[File]], product-directories, Base directories of build products.
      • exportJars, SettingKey[Boolean], export-jars, Determines whether the exported classpath for this project contains classes (false) or a packaged jar (true).
      • exportedProducts, TaskKey[Classpath], exported-products, Build products that go on the exported classpath.
      • unmanagedClasspath, TaskKey[Classpath], unmanaged-classpath, Classpath entries (deep) that are manually managed.
      • unmanagedJars, TaskKey[Classpath], unmanaged-jars, Classpath entries for the current project (shallow) that are manually managed.
      • managedClasspath, TaskKey[Classpath], managed-classpath, The classpath consisting of external, managed library dependencies.
      • internalDependencyClasspath, TaskKey[Classpath], internal-dependency-classpath, The internal (inter-project) classpath.
      • externalDependencyClasspath, TaskKey[Classpath], external-dependency-classpath, The classpath consisting of library dependencies, both managed and unmanaged.
      • dependencyClasspath, TaskKey[Classpath], dependency-classpath, The classpath consisting of internal and external, managed, and unmanaged dependencies.
      • fullClasspath, TaskKey[Classpath], full-classpath, The exported classpath, consisting of build products and unmanaged and managed, internal and external dependencies.
      • internalConfigurationMap, SettingKey[PartialFunction1[Configuration, Configuration]], internal-configuration-map, Maps configurations to the actual configuration used to define the classpath.
      • classpathConfiguration, TaskKey[Configuration], classpath-configuration, The configuration used to define the classpath.
      • ivyConfiguration, TaskKey[IvyConfiguration], ivy-configuration, General dependency management (Ivy) settings, such as the resolvers and paths to use.
      • ivyConfigurations, SettingKey[Seq[Configuration]], ivy-configurations, The defined configurations for dependency management. This may be different from the configurations for Project settings.
      • moduleSettings, TaskKey[ModuleSettings], module-settings, Module settings, which configure a specific module, such as a project.
      • unmanagedBase, SettingKey[File], unmanaged-base, The default directory for manually managed libraries.
      • updateConfiguration, SettingKey[UpdateConfiguration], Configuration for resolving and retrieving managed dependencies.
      • ivySbt, TaskKey[IvySbt], ivy-sbt, Provides the sbt interface to Ivy.
      • ivyMOdule, TaskKey[IvySbt#Module], ivy-module, Provides the sbt interface to a configured Ivy module.
      • update, TaskKey[UpdateReport], update, Resolves and optionally retrieves dependencies, producing a report.
      • transitiveUpdate, TaskKey[Seq[UpdateReport]], transitive-update, UpdateReports for the internal dependencies of this project.
      • updateClassifiers, TaskKey[UpdateReport], Resoves and optionally retrieves classified artifacts, such as javadocs and sources, for dependency definitions, transitively.
      • transitiveClassifiers, SettingKey[Seq[String]], transitive-classifiers, List of classifiers used for transitively obtaining extra artifacts for sbt or declared dependencies.
      • updateSbtClassifiers, TaskKey[UpdateReport], update-sbt-classifiers, Resolves and optionally retrieves classifiers, such as javadocs and sources, for sbt, transitively.
      • publishConfiguration, TaskKey[PublishConfiguration], publish-configuration, Configuration for publishing to a repository.
      • publishLocalConfiguration, TaskKey[PublishConfiguration], publish-local-configuration, Configuration for publishing to the local repository.
      • deliverConfiguration, TaskKey[DeliverConfiguration], deliver-configuration, Configuration for generating the finished Ivy file for publishing.
      • deliverLocalConfiguration, TaskKey[DeliverConfiguration], deliver-local-configuration, Configuration for generating teh finished Ivy file for local publishing.
      • makePomConfigurations, SettingKey[MakePomConfiguration], make-pom-configuration, Configuration for generating pom.
      • packagedArtifacts, TaskKey[Map[Artifact, File]], packaged-artifacts, Packages all artifacts for publishing and maps the Artifact definition to the generated file.
      • publishMavenStyle, SettingKey[Boolean], publish-maven-stye, Configures whether to generate and publish a pom (true), or Ivy file(false).
      • credentials, TaskKey[Seq[Credentials]], credentials, The credentials to use for updating and publishing.
      • makePom, TaskKey[File], make-pom, Generates a pom for publishing when publishing Maven-style.
      • deliver, TaskKey[File], deliver, Generates teh Ivy file for publishing to a repository.
      • deliverLocal, TaskKey[File], deliver-local, Generates the Ivy file for publishing to the local repository.
      • publish, TaskKey[Unit], publish, Publishes artifacts to a repository.
      • publishLocal, TaskKey[Unit], publish-local, Publishes artifacts to the local repository.
      • pomExtra, SettingKey[NodeSeq], pom-extra, Extra XML to insert into the generated POM.
      • pomPostProcess, SettingKey[PartialFunction1[XNode, XNode]], pom-post-process, Transforms the generated POM.
      • pomIncludeRepository, SettingKey[PartialFunction1[MavenRepository, Boolean]], pom--include-repository, Selects repositories to include in the generated POM.
      • pomAllRepositories, SettingKey[Boolean], pom-all-repositories, If true, includes repositories used in module configurations in the pom repositories section. If false, only the common repositories are included.
      • moduleName, SettingKey[String], module-name, The name of the current module, used for dependency management.
      • version, SettingKey[String], version, The version/revision of the current module.
      • isSnapshot, SettingKey[Boolean], is-snapshot, True if the version of the project is a snapshot version.
      • moduleID, SettingKey[ModuleID], module-id, A dependency management descriptor. This is currently used for associating a ModuleID with a classpath entry.
      • projectID, SettingKey[ModuleID], project-id, The dependency management descriptor fo rthe current module.
      • externalResolvers, TaskKey[Seq[Resolver]], external-resolvers, The external resolvers for automatically managed dependencies.
      • projectResolver, TaskKey[Resolver], project-resolver, Resolver that handles inter-project dependencies.
      • fullResolvers, TaskKey[Seq[Resolver]], full-resolvers, Combines the project resolver, default resolvers, and user-defined resolvers.
      • otherResovers, SettingKey[Seq[Resolver]], other-resolvers, Resolvers not included in the main resolver chain, such as those in module configurations.
      • moduleConfigurations, SettingKey[Seq[ModuleConfiguration]], module-configurations, Defines module configurations, which override resolvers on a per-module basis.
      • retrievePattern, SettingKey[String], retrieve-pattern, Pattern used to retrieve managed dependencies to the current build.
      • retrieveConfiguration, SettingKey[Option[RetrieveConfiguration]], retirieve-configuration, Configures retrieving dependencies to the current build.
      • ivyPaths, SettingKey[IvyPaths], ivy-paths, Configures paths used by Ivy for dependency management.
      • libraryDependencies, SettingKey[Seq[ModuleID]], library dependencies, Declares managed dependencies.
      • projectDependencies, TaskKey[Seq[ModuleID]], project-dependencies, Inter0project dependencies.
      • ivyXML, SettingKey[NodeSeq], ivy-xml, Defines inline Ivy XML, for configuring dependency management.
      • ivyScala, SettingKey[Option[IvyScala]], ivy-scala, Configures how Scala dependencies are checked, filtered, and injected.
      • ivyValidate, SettingKey[Boolean], ivy-validate, Enables/disables Ivy validation of module metadata.
      • ivyLoggingLevel, SettingKey[UpdateLogging.Value], ivy-logging-level, The logging level for updating.
      • publishTo, SettingKey[Option[Resolver]], publish-to, The resolver to publish to.
      • artifacts, SettingKey[Seq[Artifact]], artifacts, The artifact definitions for the current module.
      • projectDescriptors, TaskKey[Map[ModuleRevisionId, ModuleDescriptor]], project-descriptors, Project dependency map for the inter-project resolver.
      • retrieveManaged, SettingKey[Boolean], retrieve-managed, If true, enables retrieving dependencies to the current build. Otherwise, dependencies are used directly from the cache.
      • managedDirectory, SettingKey[File], managed-directory, Directory to which managed dependencies are retrieved.
      • classpathTypes, SettingKey[Set[String]], classpath-types, Artifact types that are included on the classpath.
      • publishArtifact, SettingKey[Boolean], publish-artifact, Enables (true) or disables (false) publishing an artifact.
      • packagedArtifact, TaskKey[Pair[Artifact, File]], packaged-artifact, Generates a packaged artifact, returning the Artifact and the produced File.
      • checksums, SettingKey[Seq[String]], checksums, The list of checksums to generate and to verify for dependencies.
      • classifiersModule, TaksKey[GetClassifiersModules], classifiersModule, NA
      • conflictWarning, SettingKey[ConflictWarning], conflict-warning, Configures warnings for conflicts in dependency management.
      • autoScalaLibrary, SettingKey[Boolean], auto-scala-library, Adds a dependency on scala-library if true.
      • sbtResolver, SettingKey[Resolver], sbt-resolver, Provides a resolver for obtaining sbt as a dependency.
      • sbtDependency, SettingKey[ModuleID], sbt-dependency, Provides a definition for declaring the current version of sbt.
      • sbtVersion, SettingKey[String], sbt-version, Provides the version of sbt. This setting should not be modified.
      • sbtBinaryVersion, SettingKey[String], sbt-binary-version, Defines teh binary compatibility version substring.
      • skip, TaskKey[Boolean], skip, For tasks that support it (currently only 'compile'), setting skip to true will force the task to not do its work. This exact semantics may vary by task.
      Special Settings
      Format (key name, key type, sbt command-line key name, description)

      • sessionVars, AttributeKey[SessionVar.Map], session-vars, Bindings that exist for the duration of the session.
      • parallelExecution, SettingKey[Boolean], parallel-execution, Enables (true) or disables (false) parallel execution of tasks.
      • tags, SettingKey[Seq[Pair[Tags.Tag, Int]]], tags, ConcurrentRestrictions.tagsKey.label
      • concurrentRestrictions, SettingKey[Seq[Tags.Rule]], concurrent-restrictions, Rules describing restrictions on concurrent task execution.
      • cancelable, SettingKey[Boolean], cancelable, Enables (true) or disables (flase) the ability to interrupt task execution with CTRL+C.
      • settings, TaskKey[Settings[Scope]], settings, Provides access to the project data for the build.
      • streams, TaskKey[TaskStreams], streams rovides streams for logging and persisting data.
      • isDummyTask, AttributeKey[Boolean], is-dummy-task, Internal: used to identify dummy tasks. sbt injects values for these tasks at the start fo task execution.
      • taskDefinitionKey, AttributeKey[ScopedKey[_]], task-definition-key, Internal: used to map a task back to its ScopedKey.
      • resolvedScoped, SettingKey[ScopedKey[_]], resolved-scoped, The ScopedKey for the referencing setting or task.




                        Tuesday, January 10, 2012

                        SBT ~ Scala's Simple Build Tool Series, Part 1. What is SBT and how do I use it?

                        Scala's Simple Build Tool is an all-in-one plug-n-play build and dependency management tool. Despite the name, SBT is often criticized for being difficult to use and extend, and for marginalizing the traditional Java ecosystem build tools - Maven and Ant. I too have had some difficulty understanding the internal concepts of SBT, so I decided to write a series of blog posts on the topic, both to help myself gain a deeper understanding and to help others who are struggling with understanding the tool.


                        What SBT is:
                        SBT is the build tool of choice for Scala projects. Like other build tools for other languages SBT's purpose is to compile and publish applications written in Scala. It is a command-line tool written in scala. It defines both an API and a Domain Specific Language (DSL) for building and testing scala projects, managing dependencies, publishing projects, and running them locally. While the SBT Getting Started Guide is helpful for getting off of the ground, it can be a little difficult to grok, and will probably take several readings to fully understand. Additionally, you are probably going to have to read the SBT source code to really get how to use SBT to its fullest potential. The SBT API documentation is a good resource for looking up method signatures and predefined value names, but it lacks a blow-by-blow explanation of what those methods do or what they expect as parameters beyond basic type definitions for parameters and returns. 


                        This can be troublesome, because many of the methods are symbols (Scala allows methods like ~= and &lt;&lt;=) or require understanding of some basic Scala concepts like defining and applying anonymous functions, strongly typed generic collections,  and traits that a newly minted user of Scala might not fully understand. As always, reading the source code is the best way to really understand an application, and I encourage you to do so. Also, as the project is open source, please feel free to contribute better documentation and examples by forking the xsbt github project. I hope to provide here a companion guide to SBT that will deepen my own and others' understanding of SBT, as well as some helpful hints for getting started with SBT.


                        SBT Requirements:
                        1. JVM 1.5 or later.
                        2. The latest scala distro.
                        3. The latest sbt-launch.jar.
                        4. Paul Phillips' excellent sbt-extras sbt launch script.
                        Installation:
                        1. Download and install sbt-launch.jar and place it in ~/bin. DO NOT put sbt-launch.jar in your CLASSPATH. You don't need to do that.
                        2. Clone git://github.com/paulp/sbt-extras.git.
                        3. cd into the cloned sbt-extras repo and copy sbt to ~/bin.
                        4. chmod u+x ~/bin/sbt
                        That's it! Congratulations, you have installed SBT.

                        A Simple Hello World! project:
                        Now let's set up a simple Hello World scala example. SBT expects a certain project structure. For historical reasons, this project structure is similar to the project structure used by Apache Maven projects:

                        lib/
                           <unmanaged jars go here>
                        src/
                           main/
                              resources/
                                 <files to include in project jar go here>
                              scala/
                                 <main scala source files go here>
                              java/
                                 <main Java source files go here>
                           test/
                              resources/
                                 <files to be included in the project test jar go here>
                              scala/
                                 <scala test source files go here>
                              java/
                                 <Java test source files go here>
                        project/
                                 <project build scala files go here>


                        1. Create a helloProject directory
                        2. cd to helloWorld
                        3. mkdir lib
                        4. mkdir -p src/main/resources
                        5. mkdir src/main/scala
                        6. mkdir src/main/java
                        7. mkdir -p src/test/resources
                        8. mkdir src/test/scala
                        9. mkdir src/test/java
                        10. mkdir project

                        Now that you have defined your project structure, let's create our first .sbt build definition file. Open your favorite text editor and enter the following, including the empty lines:


                        name := "helloWorld"


                        version := "1.0"


                        scalaVersion := "2.9.1"


                        Save the file as helloWorld/build.sbt.


                        build.sbt Explanation
                        build.sbt is a SBT build definition file. Build definition files contain Scala expressions and function definitions using the SBT DSL. What you have just defined is a list of settings that SBT will use when building and running your projects.


                        Remember when I said that you would need to understand a few basic Scala concepts to understand SBT? Well here is where you get to learn about a few of them. Each line of the build.sbt file above is Scala code. Each line is a separate Scala expression that will be applied to the default list of settings to produce a new list of settings that SBT will use to build your project. The newlines are delimiters used by SBT to separate each expression. The identifiers on each line (name, version, etc.) represent predefined sbt.Keys.  The := on each line calls the sbt.Keys.:= method with the argument string that follows it. The := returns a generically typed setting of type Setting[T]. Since all of our settings are strings, our build.sbt defines a list of Setting[String] objects.


                        Now, since we are using the extremely helpful sbt-extras sbt script, we need to define a build.properties file in our project's project directory. Fire up your favorite text editor again, and enter the following text:


                        sbt.version=0.11.2


                        Save the file as helloWorld/project/build.properties.


                        build.properties Expanation
                        project/build.properties is used by the sbt-extras sbt script to decide which version of sbt to use to build your project. The script will download the appropriate version of sbt and execute it to build your project.


                        Finally, lets create a simple Scala application that will print "Hello World!". Fire up the text editor again and enter the following Scala code:


                        object Hello{
                           def main(args: Array[String]) = println("Hello World!)
                        }


                        Save the file in helloWorld/src/main/scala/HelloWorld.scala.


                        Now we're ready to run sbt for the first time. Go to the console, cd into the helloWorld directory, and type sbt. After the prompt, type run. The sbt console will run, download the necessary dependencies (the correct scala version, for example) and print "Hello World!". Type exit to leave sbt.


                        That's it. Now you know how to define a project to use sbt with a simple build definition. Hopefully, this helps you get started.


                        Next Article:
                        The next article will focus on the different types of Settings provided by SBT out of the box. Happy Hacking!


                        Credits:

                        harrah/xsbt. 2012 Jan 10. _root_.
                           harrah / xsbt. <http://harrah.github.com/xsbt/latest/api/index.html#package>. Accessed 2012 Jan 10.

                        harrah/xsbt. 2012 Jan 10. harrah/xsbt.
                           harrah/xsbt. <https://github.com/harrah/xsbt>. Acessed 2012 Jan 10.


                        harrah/xsbt. 2012 Jan 10. Directory structure.
                           harrah/xsbt. <https://github.com/harrah/xsbt/wiki/Getting-Started-Directories>. Accessed 2012 Jan 10.


                        harrah/xsbt. 2012 Jan 10. Setup.
                           harrah/xsbt. <https://github.com/harrah/xsbt/wiki/Getting-Started-Setup>. Accessed 2012 Jan 10.

                        harrah/xsbt. 2012 Jan 10. Setup Notes.
                           harrah/xsbt. <https://github.com/harrah/xsbt/wiki/Setup-Notes>. Accessed 2012 Jan 10.


                        harrah/xsbt. 2012 Jan 10. Welcome!.
                           harrah/xsbt. <https://github.com/harrah/xsbt/wiki/Getting-Started-Welcome>. Accessed 2012 Jan 10.


                        Phillips, Paul. 2012 Jan 10. paulp/sbt-extras.
                           paulp/sbt-extras. <https://github.com/paulp/sbt-extras>. Accessed 2012 Jan 10. 


                        van Zyl, J. 2012 Jan 10. Introduction to the Standard Directory Layout.
                           Apache Maven Project. <http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html>. Accessed 2012 Jan 10.

                        Tuesday, June 14, 2011

                        Re: Why Today Web Development Sucks

                        Stumbled upon a post by Harry Brundage about how web development sucks today. His main argument stems from the fact that Single Page Applications (SPA) and web stacks tend to suffer from violations of the DRY principle. Harry makes some good points, and proposes a single language framework that would bring something like Ruby On Rails to the JavaScript domain. While this may be good, I think the problem is in the way applications are conceived of and architected. A properly architected solution can avoid most of the DRY violations Harry speaks of, and will help you to do web development that doesn't suck, today.


                        Web Development is Hard
                        The crux of the issue is that developing a web application is hard. Developing a SPA is harder. At first glance, building a web application seems simple enough: write some HTML and CSS, put in a few JavaScript DOM event hooks, create a few web services in your language of choice, expose those web services via HTTP, and voila, you have a web application.


                        However, that simple description isn't simple at all, is it? I mean, we have a markup language and a styling language to learn, a framework (DOM) to learn and understand, an interpreted execution language to learn and master, a server-side language to learn and master, and a stateless protocol to allow data transport between the client and server. Then we've got security, persistence, browser differences, performance, download times, SEO, caching, maintainability and scalability concerns to add on top of the prerequisite knowledge stated previously. Then we have software architecture to learn. That's a lot of knowledge for one person to learn and use well.


                        Then we have the users and the client/customer to worry about. Oh, and application GUI design, because nobody wants a web application to look like some old Windows Forms application brought to the web. Animation, effects, transitions – there is more than enough to learn here for someone to specialize in their entire career and not be out of work. That's often what happens. However, acceptance and understanding of a few key concepts can help you get a handle on all of these things to become a good, general web application developer.


                        Learn To Think in Layers of Responsibility
                        In order to make sense out of this mess, I find it easier to lay out an application in layers of responsibility. These layer patterns are often recursive, and repeated within themselves. In its simplest terms, every layer should abide by the Unix Principles that a layer should "do one thing and do it well" and that a layer should be "written to work with other layers." I find this to be true for nearly all applications, but it especially applies to web applications and even more to SPAs.


                        Nearly every web application has at least three basic layers. The first is a persistence layer. This layer is where all your data is stored between requests from the client to the server. It stores the state of your entire application. Second, is the service layer. This layer provides communication and data formatting for requests from the client to the persistence layer. Finally, the only part of the application that your users will see, comes the client layer, which transforms the data into a human usable format using HTML, CSS, and JavaScript.


                        Notice that I did not use the traditional Model, View, Controller (MVC) pattern to describe the persistence layer as the Model, the service layer as the Controller, and the client as the View. Modern web applications, and SPAs in particular, have moved beyond this basic concept.


                        In reality, each layer will probably have its own MVC pattern, in which the persistence layer's model is the physical data structure, its controller the mechanism for reading and writing to that structure (typically a database engine with SQL, though XML, file-system, and NoSQL map-reduce schemes are popular as well), and its view the format of the data return (often handled by your language of choice's database connection library).


                        The service layer will likely have these features as well, with sections of code containing a business or domain model that is mapped to the persistence view, a controller for translating client models to and from the business model for manipulation, processing, and storage to the persistence layer, and a view section that formats data coming into and out of the client layer.


                        Likewise, a client will have an in-memory (typically, disregarding cookies and local storage) model for communicating with the business layer, a controller for processing communications from the client's view and from the service layer to the client, and a view (HTML, CSS, DOM, JavaScript GUI toolkit of choice) to present the data to the client.


                        The key thing is that each layer presents a view to the other layer, which processes it, and in turn presents the view to the next layer, and so on. The layers communicate with each other in ways that each layer can understand. The persistence layer has no knowledge of the client layer. The service layer need not have knowledge of how the client layer will present the data to the user, so long as the service layer provides a view that the client can interpret. The client layer need not have any explicit knowledge of the persistence layer, other than that the data must be retrieved and stored. If you learn to think in layers like this, then you can begin to separate the features and responsibilities of the layers involved, thus simplifying the responsibilities of each layer.


                        DRY Web Apps
                        The key thing to avoid repeating yourself, and thus save development headaches down the road when maintaining and upgrading an application, is to separate the responsibilities of the business logic between the layers. A good way I've found to do that is to model the whole stack as a series of Objects as if the entire web stack were to be coded as a single application. Treat each layer as a Facade to its inner workings. Then you only have three interfaces to work with! A persistence interface, a service interface, and a client interface. Strongly type the parameters and returns from methods on each layer. Treat communication objects and returns from each layer as Data Transfer Objects (DTO). This separates the models on each layer, preventing interference.


                        Validate only when and to the degree necessary. If you want to alert users that they aren't typing in their MasterCard in the correct format, you can validate the format using regular expressions on the client side. Yes, you'll have to duplicate the regex on the server side, but could you possibly persist the regular expression somewhere in your application for use by both layers? That would prevent you from having to maintain it in two layers. Also, simply validating the format of data isn't truly validating the credit card information is it? You have to combine the Name and Billing Address as well. If you are not storing the card, there isn't any need to strip out things for SQL injection is there? And since all of this business logic has to do with validation, perhaps a single validation Facade for all your exposed services would be a good place to store all that logic?


                        By using good object oriented design principles while modeling your application, you can avoid repeating yourself everywhere, and keep the business necessary to the appropriate layer in that layer, and keep the layer transparent to the other layers. The other advantage of modeling your application in a unified model is that it keeps you out of the weeds of the particular programming languages involved in the implementation of the model. Which brings me to my next point regarding transferring data between layers.


                        Be Language Agnostic
                        When you format your views from each layer to the next, it is important to keep the format of the data to be transferred language agnostic. Don't use binary data formats that only communicate with one client side language type to transfer data into and out of the service layer. If the browsers implement Objective-C as a client-side runtime, you may want to be able to use that. JSON and XML work particularly well for transferring information because they are string based and have strict syntactical rules. HTML does not work so well for transferring information because it doesn't have strict syntactical rules. There should be NO HTML in your returns or accepted data formats. You never know what language of a public service API that developers will use to connect to your services, so keep the information in some form of a string-based implementation.


                        Be Consistent
                        Be consistent with your data format returns. If you choose to use Comma-Separated-Values(CSV) for one service, do it for all of them. If you can't, use JSON instead. If you need references to other nodes in your data return, use XML. The key thing is to use a consistent format. You don't want to write serializers for a multitude of different possible data formats on the client side or the server side. It will lead to repeated code and a lot of difficult corner cases.


                        Be General and Concise
                        Try to be general with your services and returns. Don't write a thousand services for every possible request. Write as few general services as you can to relate any data to any other related data and generally expose more information rather than less. The most expensive part of a web application is requesting information from another layer, so try to limit those up front with general services. If you run into trouble later on down the road or your returns begin to take too long and you've exhausted every other avenue of optimization, then create a new service.


                        If Language Is a Barrier...
                        Use JavaScript for your server-side and client-side language of choice. If you are building a web application, JavaScript is the one language that can be used at runtime on the client-side and on the server-side. Node.js is a great place to start with server-side JavaScript. There are a ton of tutorials on getting started. CommonJS is an organization working to create a standardized API for writing JavaScript outside the browser. I suggest you check them out if you don't have time/feel like learning one language for the client side and one for the server side.


                        Writing web applications may suck today, but it doesn't have to tomorrow. If you use good object oriented design patterns, separate your application design into layers of responsibility, and manage your layers as if they were objects on the same system communicating with each other rather than separate systems over the wire, you can make it suck less. You can avoid some of the language syntax issues as well if you use Node.js and JavaScript on the server side. All of this is easy enough to do with enough planning, and you'll have a much happier day playing around with your application. Happy Coding!