Continuous Integration

Continuous integration automates the building, testing and deploying of applications. Software projects, whether created by a single individual or entire teams, typically use continuous integration as a hub to ensure important steps such as unit testing are automated rather than manual processes.

Why is continuous integration important?

When continuous integration (CI) is established as a step in a software project's development process it can dramatically reduce deployment times by minimizing steps that require human intervention. The only minor downside to using CI is that it takes some initial time by a developer to set up and then there is some ongoing maintainence if a project is broken into multiple parts, such as going from a monolith architecture to microservices.

Automated testing

Another major advantage with CI is that testing can be an automated step in the deployment process. Broken deployments can be prevented by running a comprehensive test suite of unit and integration tests when developers check in code to a source code repository. Any bugs accidentally introduced during a check-in that are caught by the test suite are reported and prevent the deployment from proceeding.

The automated testing on checked in source code can be thought of like the bumper guards in bowling that prevent code quality from going too far off track. CI combined with unit and integration tests check that any code modifications do not break existing tests to ensure the software works as intended.

Continuous integration example

The following picture represents a high level perspective on how continuous integration and deployment can work.

One potential way for continuous integration to work with source control and a deployment environment.

In the above diagram, when new code is committed to a source repository there is a hook that notifies the continuous integration server that new code needs to be built (the continuous integration server could also poll the source code repository if a notification is not possible).

The continuous integration server pulls the code to build and test it. If all tests pass, the continuous integration server begins the deployment process. The new code is pulled down to the server where the deployment is taking place. Finally the deployment process is completed via restarting services and related deployment activities.

There are many other ways a continuous integration server and its deployments can be structured. The above was just one example of a relatively simple setup.

Open source CI projects

There are a variety of free and open source continuous integration servers that are configurable based on a project's needs.

Note that many of these servers are not written in Python but work just fine for Python applications. Polyglot organizations (ones that use more than a single language and ecosystem) often use a single CI server for all of their projects regardless of the programming language the application was written in.

Jenkins CI resources

Jenkins is commonly used as a continuous integration server implementation for Python projects because it is open source and programming language agnostic. Learn more via the following resources or on the dedicated Jenkins page.

General continuous integration resources

What do you want to work on next for your deployment?

How do I log errors that occur in my application?

What can I do to mitigate security vulnerability in my web app?

What is Docker and how does it fit with Python deployments?

Sign up for two emails per month with Python tutorials and Full Stack Python updates.

Sponsored By

Rollbar logo

Fix errors in your Python applications before your users ever see them by monitoring your app with Rollbar.

Full Stack Python

Full Stack Python is an open book that explains concepts in plain language and provides helpful resources for those topics.
Updates via newsletter, Twitter & Facebook.
1. IntroductionLearning ProgrammingWhy Use Python?Python 2 or 3?Enterprise PythonPython CommunityCompanies using PythonBest Python ResourcesBest Python VideosBest Python Podcasts2. Development EnvironmentsText Editors & IDEsVimEmacsSublime TextPyCharmJupyter NotebookBash shellShellsZshPowerShellTerminal MultiplexerstmuxScreenPymuxEnvironment configurationApplication DependenciesSource ControlGitMercurialApache SubversionHosted Source ControlGitHubBitBucketGitLab3. Core LanguageGeneratorsComprehensions4. TestingUnit TestingIntegration TestingCode MetricsDebuggingLoggingMarkdown6. Security7. Web DevelopmentWeb FrameworksDjangoFlaskBottlePyramidFalconMorepathSanicOther Web FrameworksTemplate EnginesJinja2MakoDjango TemplatesWeb DesignCascading Style Sheets (CSS)HTMLResponsive DesignMinificationBootstrapJavaScriptTask QueuesCeleryRedis Queue (RQ)DramatiqStatic Site GeneratorsPelicanLektorMkDocsWebSocketsuvloop8. DeploymentServersStatic ContentPlatform-as-a-ServiceVirtual Private ServersOperating SystemsUbuntuWeb ServersApache HTTP ServerNginxCaddyWSGI ServersGreen Unicorn (Gunicorn)Continuous IntegrationJenkinsConfiguration ManagementAnsibleDockerServerlessAWS LambdaGoogle Cloud Functions9. DataRelational DatabasesPostgreSQLMySQLSQLiteObject-relational MappersSQLAlchemyPeeweeDjango ORMSQLObjectPony ORMNoSQL Data StoresRedisMongoDBApache CassandraNeo4jData analysispandasBokehd3.jsMatplotlib10. Web APIsMicroservicesBotsAPI CreationAPI IntegrationTwilio11. DevOpsMonitoringCachingRollbarWeb Analytics12. Change LogWhat Full Stack MeansAbout the AuthorFuture DirectionsPage Statuses ...or view all topics.

Matt Makai 2012-2018