Pants 2.9: Alpha support for Java and Scala, improvements for Docker and Go, and more
We're pleased to announce [Pants 2.9.0](https://www.pantsbuild.org/v2.9/docs), the latest release of Pants: the scalable and ergonomic build system!
We're pleased to announce Pants 2.9.0, the latest release of Pants: the scalable and ergonomic build system!
To update, set
pants_version = "2.9.0" in your
pants.toml. See upgrade tips for more information.
Alpha Support for Java and Scala
We’re very happy to announce that support for Java and Scala has reached alpha quality in the
Pants 1.x had a long history of support for Java and Scala, going back to when it was first created at Twitter. In fact: they were the first supported languages! Consequently, (re-)adding support for these popular JVM languages has been high on our list ever since the 2.x release in late 2020.
Improvements over Pants 1.x
In the last few years, we’ve learned a lot about how best to deal with more-slowly-compiling, high level languages like Scala.
Dependency inference and per-file compilation
As described in our recent blog post, dependency inference for Java and Scala removes a ton of boilerplate.
But very fine-grained, always-accurate dependencies also enable per-file compilation, reducing the number of files that Pants needs to feed to
scalac, and allowing for automatic file-level parallelization and the most accurate cache keys possible. From a correctness perspective, that means that unlike tools which use compilation libraries like Zinc (SBT, Bloop, Mill, optionally Bazel, and others) Pants
2.9.0 supports sandboxed, minimal incremental Java and Scala compilation, while preventing the under-compilation bugs that have historically troubled Scala developers.
For more information on how dependency inference works for the JVM, check out that post!
Multiple resolves of third party dependencies
Another significant improvement for the JVM over 1.x is that Pants
2.9 implements a monorepo-friendly multiple-resolve/lockfile strategy for third party dependencies, allowing for correctness, flexibility, and performance.
Build tools for the JVM tend to either resolve dependencies globally (for an entire repository), or locally (on a project-by-project basis). Global resolves (as in Bazel) remove flexibility, because teams working within a repository cannot diverge from the single blessed versions of any library: if they try, they are nearly guaranteed to encounter classpath incompatibilities. On the other hand, local/per-project resolves (as in SBT, Gradle, Maven) within a monorepo allow for local flexibility, but reduce the performance and compatibility of any particular build by executing one unique resolve per project.
Rather than forcing global or per-project resolves, Pants
2.9 supports a unique hybrid approach: third party resolves are named and first-class, and can be used on a target by target basis. This allows a monorepo to operate with the minimum number of resolves required to support their conflicting library versions, without necessarily going to the costly extreme of per-project resolves.
Having the minimum number of resolves improves performance, but it doesn’t come with a correctness cost! To ensure reproducible builds, Pants generates a lockfile per resolve, which is then efficiently consumed to fetch the precise, fingerprinted dependencies of any particular file.
Trying out Java or Scala
Although this is an alpha release, the features that are implemented so far are expected to give teams enough to work with to validate using Pants with a JVM codebase:
- Use of nailgun to keep warm JVMs for compilers and tools
- ScalaTest support for Scala, and JUnit support for both Java and Scala
- Scalafmt and Google Java Format
- Scala Repl support
- Protobuf code generation with ScalaPB
- Debugging support via
./pants test --debug $file
- Scala compiler plugins
- Support for compiling cycles between Java and Scala code
- Bootstrapping a consistent JVM using coursier
- Multiple first class thirdparty resolves (coursier again!) with independent lockfiles
We’d love to help you try out Pants with your JVM codebase: you can start by checking out the initial documentation and example JVM repository. If you see anything missing that prevents you from evaluating Pants for Java and Scala, please let us know by opening a Github issue, or visiting Slack!
Better visibility into runtime and caching for tests
Thanks to a great change from a new contributor, Pants
2.9 now renders test runtime and cache status (whether a process ran, hit a cache, or was memoized in memory by
pantsd) in the test summary for all supported languages!
❯ ./pants test src/python/pants/util: … ✓ src/python/pants/util/dirutil_test.py:tests succeeded in 1.21s (cached locally). ✓ src/python/pants/util/osutil_test.py:tests succeeded in 0.72s (memoized). ✓ src/python/pants/util/strutil_test.py:tests succeeded in 0.98s (cached remotely).
No more thinking to yourself: “Gee, that was even faster than usual! I wonder why?”
Improvements to Docker support
Among a number of bug fixes and documentation enhancements, there were a few noteworthy improvements to the Docker backend:
- Introduce a new
target_stagefield for the
docker_imagetarget as well as the
docker_imageto support generating an anonymous Dockerfile from scratch.
- Log suggestions of how to fix the docker build context when a docker build fails.
- New option for the
[docker]scope, which allows passing additional options when executing
docker run [OPTIONS] <image>, in addition to the
--run-argswhich are passed to the image entrypoint.
- For example:
./pants run --docker-run-args="-p 8080/tcp" src/docker:example -- --arg-for-the-image-entrypoint
- For example:
- Add new
- Support interpolating Docker build args into the
- Allow tailor to create
docker_imagetargets for any files with “Dockerfile” in the file name.
Check out the updated
docker documentation for more information!
Support for running
./pants tailor goal automatically updates
BUILD files to add targets for recognized file types, which makes it great for onboarding a new repository to Pants.
tailor's options to allow it to be run on an ongoing basis in your repository, to ensure that your
BUILD metadata stays up to date as files are added.
To do this,
tailor gained a
--check mode, and now supports ignoring files which should be visible to Pants (i.e., not in pants_ignore), but not have
BUILD targets generated for them.
Changes to Go project layouts
After adding experimental support for Go in Pants
2.8, we decided that a few changes to how Go targets are laid out in BUILD files would help to future proof the support.
To that end, in Pants
2.9, each Go package now needs its own
go_package declaration in a
BUILD file. Thanks to dependency inference and
./pants tailor, these BUILD files are very simple, can be created automatically, and rarely require adjustments. But when you do need to add metadata, the new layout follows the 1:1:1 principle: metadata about your code should live near the code itself.
New Go features
In particular, the changes to target layouts make it easier to use the new Go features in
go_package targets now support:
1. Setting a
test_timeout for the package (in seconds), which is propagated down to the Go test runner:
go_package( test_timeout=120, )
resource files in a binary for use at runtime:
go_package(dependencies=[":resources"]) resources( name="resources", sources=["*.txt"], )
files to the working directory of your tests (i.e.
go_package(dependencies=[":testdata"]) file( name="testdata", source="testdata/f.txt" )
Updating for Go
To update your
BUILD files for the new layout in
2.9, all you need to do is run
./pants tailor. Thanks for your patience, and for all of your feedback on the new Go support! Please continue to let us know how it can be most useful to you.
Thanks to all the contributors to
2.9, including everyone who shared feedback on changes and who tested release candidates!