Goal.
This project is about getting the best of Docker and Jenkins to offer a simple, efficient CI/CD infrastructure. This project is highly opinionated, so you might disagree :P
Embrace Docker.
Docker and Jenkins at some time overlap in the way they can manage build processes and resources. Our goal here is to explore Docker impact on Jenkins design to benefit Docker's features when they do a better job.- Jenkins uses it's own mechanisms to control build processes on remote "agents" (aka "slaves"). Docker API allows to control remote processes as well. We prefer the later. Actually, we have plan to fully remove need for Jenkins remoting
- Docker offer isolated, customizable environment to run process(es). We want end-user to get total flexibility on this.
- Docker can run containers as a composed set of services (aka docker-compose or kubernetes pods). We want to rely on this for end-user to compose a build/test environment for CI/CD, and not consider a container as a lightweight virtual machine.
- Docker offer volumes and volume-drivers to manage data persistence and/or replication with low level control on actual infrastructure. We want to rely on this for build workspace, vs letting Jenkins guess the best node to run build on.
Docker executor.
docker-slaves-plugin is our first attempt to achieve some of those goals. It runs a set of ephemeral docker containers to host the build, based on configuration set by end user directly in job configuration.
This project is released, not strictly speaking "ready for production" as there's some glitches we'd like to address and some extension point to be defined. Anyway it is stable enough so hundred people already adopted it :PDocker-slaves relies on docker composability to run a set of containers with shared resources (i.e. workspace and network). We use this to run jenkins remote agent plumbing in a container end-user don't have to wory about, in addition to build container that can run arbitrary image, as well as "side containers" to provide test resources (test database, selenium headless browser, ...)
This plugin has a high potential as it is fully compatible with all Jenkins plugins, do support Jenkins Pipeline, and could in near future support docker-compose syntax as well as run on docker orchestrator (kubernetes or mesos for sample) in addition to plain docker/swarm.
Docker Pipeline.
docker-slaves-plugin do support pipeline, but still relies on executing arbitrary pipeline steps, and as such on jenkins remoting. We plan to offer an alternative pipeline-specific approach to only support shell commands, and run container detached. Doing this, we can just wait for the container to stop (listening on docker events to get notified) and don't need long-running-task plugin.
Main idea of this plugin is to rely only on Docker, and drop Jenkins remoting. This make it fully incompatible with other Jenkins plugins, so we will have to re-implement the relevant CI/CD features. Sounds crazy, but we think it's the best way to explore "how Jenkins would look if Docker existed when it was created".
It might be possible to be some way compatible with Jenkins plugins, by exposing some remoting stub to plugins, which would only support a subset
of Jenkins remoting API, typically re-implement FilePath remote file abstraction with plain docker commands (typically, docker cp).
This is just an assumption so far, doing this is a huge effort, probably simpler for us to define our own API and offer plugin adapters for the most requested ones.
This plugin is in early proof-of-concept stage.
See draft design doc
Declarative Pipeline.
A longer terms goal is to offer a declarative pipeline. As pipeline let end-user do arbitrary groovy scripting, it require sandboxing and increased complexity. This can make sense for some use-cases but we want to offer a KISS-level pipeline experience, inspired by travis-ci|circle-ci|drone.io. We can model pipeline stages as docker container(s) running some commands.
By design, such a DSL pipeline would offer limited extensibility. This would let us gain full control on the set of supported features, and would make it possible to just translate into docker-pipeline Groovy DSL code to benefit Jenkins Pipeline existing infrastructure and active development effort.
see draft design doc