Monitoring tools capture, analyze and display information for a web application's execution. Every application has issues arise throughout all levels of the web stack. Monitoring tools provide transparency so developers and operations teams can respond and fix problems.
Capturing and analyzing data about your production environment is critical to proactively deal with stability, performance, and errors in a web application.
Monitoring and logging are very similar in their purpose of helping to diagnose issues with an application and aid the debugging process. One way to think about the difference is that logging happens based on explicit events while monitoring is a passive background collection of data.
For example, when an error occurs, that event is explicitly logged through code in an exception handler. Meanwhile, a monitoring agent instruments the code and gathers data not only about the logged exception but also the performance of the functions.
This distinction between logging and monitoring is vague and not necessarily the only way to look at it. Pragmatically, both are useful for maintaining a production web application.
There are several important resources to monitor on the operating system and network level of a web stack.
Application level monitoring encompasses several aspects. The amount of time and resources dedicated to each aspect will vary based on whether an application is read-heavy, write-heavy, or subject to rapid swings in traffic.
Hosted monitoring software takes away the burden of deploying and operating the software yourself. However, hosted monitoring costs (often a significant amount of) money and take your application's data out of your hands so these services are not the right fit for every project.
Application Performance Monitoring (APM)
How to Add Hosted Monitoring to Flask Web Applications and How to Monitor Bottle Web Applications are a couple of posts in a series showing how to add hosted monitoring to Python web apps built with any of the major Python web frameworks.
Practical Guide to StatsD/Graphite Monitoring is a detailed guide with code examples for monitoring infrastructure.
Bit.ly describes the "10 Things They Forgot to Monitor" beyond the standard metrics such as disk & memory usage.
5 years of metrics and monitoring is a great presentation highlighting that visualization so humans can understand measurements is a hard problem. Line graphs are often not the best solution and they are overused.
The Collector Highlight Series has an article on StatsD that explains how to install it and how it works.
This survey on monitoring tools has some nice data and graphs on what developers and operations folks use in their environments.
Ryan Frantz wrote a nice post on Solving Monitoring with a new definition of what monitoring means based on today's complex systems and how the practice should evolve going forward.
Review the software-as-a-service and open source monitoring tools below. Third party services tend to be easier to set up and host the data for you. Open source projects give you more control but you'll need to have additional servers ready for the monitoring.
My recommendation is to install New Relic's free option with the trial period to see how it works with your app. It'll give you a good idea of the capabilities for application-level monitoring tools.
As your app scales take a look at setting up one of the the open source monitoring projects such as StatsD with Graphite. The combination of those two projects will give you fine-grained control over the system metrics you're collecting and visualizing.