We approach quality assurance from many angles. A web site with a content management system behind it requires a comprehensive test plan.
Theme/Template Layer QA
Theme QA is a review between design/user experience, business owners, and Bear Group to make sure that what was designed was implemented. Photoshop is not HTML, so iterative adjustments may be needed once the site is built. We spend a lot of time at this phase, especially for mobile sites doing cross-browser testing, and we typically use Browserstack to try many different combinations of browser, operating system, and device. This is the area in which we see the most bugs or improvements during testing, but they are also the quickest to resolve.
Site-building QA is a highly iterative process of taking the design and site map and teasing out the administration/CMS components needed for content types, fields, navigation, views, taxonomy, and more. Adjustments are quick to make, and happen throughout the build process with input from editors, front-end developers and the technical project manager. For modules that have complex configuration, for instance the Workbench workflow module that creates many scenarios for the publishing process, we often will use more formal QA methods.
E-commerce sites pose tangible QA challenges, and we complete end-to-end testing for each site. This means that we place live credit cards through live gateways, and verify that funds are visible in the client’s bank account before sites go live.
In a CMS, you can easily change all text and images, so content issues are never really bugs—they are revisions. They can often be addressed immediately by copy editors or others using the system. The CMS workflow process has two steps for checks prior to publication of content, so the person with Publishing keys ultimately has the QA responsibility.
Custom Development QA
As new modules are written, there is always a detailed QA process to verify that the functionality is correct. This QA process is especially important for any integrations that introduce system interdependency. Professional testers lead this QA process, which typically has written supporting documentation and test specs.
Deployment & Build Verification Testing
The deployment process—moving code from staging environments to live production— is one of the highest-risk tasks for any web site. Our development environments encourage best practices and safe places for developers to work out the details.
Our code is all stored in version-control systems, generally Git, for easy rollback. We track code with tickets.
Developers work in local environments and commit code to our repository, where it is pushed to a staging server for testing and client review
We use automated build systems and dev/stage/production tiers with exact replica environments
We use a hands-free, repeatable process for deployment so it is reliable with several check-and-balance points
However, a code update on production can still trigger database updates, permission issues, mis-configuration, caching issues… many ways for things to break in a highly visible way. We use deployment verification tests—usually manual—to spot-check the important parts of the web site just after deployment to ensure that all key areas of the site are functioning after deployment.